[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 22:58:38 dist-test-slave-f63b krb5kdc[3003](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 22:58:38 dist-test-slave-f63b krb5kdc[3003](info): set up 2 sockets
Jun 20 22:58:38 dist-test-slave-f63b krb5kdc[3003](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260620 22:58:39.960214 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.007s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 22:58:40 dist-test-slave-f63b krb5kdc[3003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T22:58:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T22:58:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
I20260620 22:58:40.134688 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:46045
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:41123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:46045
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:58:40.459815 3016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:58:40.460101 3016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:58:40.460732 3016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:58:40.467859 3016 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 22:58:40.467972 3016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:58:40.468075 3016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:58:40.468180 3016 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 22:58:40.468250 3016 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 22:58:40.477968 3016 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:41123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:46045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:46045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:58:40.479342 3016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:58:40.480695 3016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:58:40.492993 3021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:40.493048 3022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:40.493075 3024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:58:40.493613 3016 server_base.cc:1034] running on GCE node
I20260620 22:58:40.494550 3016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:58:40.496126 3016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:58:40.497356 3016 hybrid_clock.cc:648] HybridClock initialized: now 1781996320497287 us; error 60 us; skew 500 ppm
I20260620 22:58:40.497695 3016 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:58:40 dist-test-slave-f63b krb5kdc[3003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996320, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:58:40.501121 3016 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 22:58:40.502724 3016 webserver.cc:460] Webserver started at http://127.2.237.62:39679/ using document root <none> and password file <none>
I20260620 22:58:40.503393 3016 fs_manager.cc:359] Metadata directory not provided
I20260620 22:58:40.503540 3016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:58:40.503844 3016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:58:40.506891 3016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "809781b7e9134b45b24e52b6d48bef8f"
format_stamp: "Formatted at 2026-06-20 22:58:40 on dist-test-slave-f63b"
server_key: "5d896337cdf164319071c541e64c5210"
server_key_iv: "1a63c739039b80ba11704bc496853309"
server_key_version: "encryptionkey@0"
I20260620 22:58:40.507627 3016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "809781b7e9134b45b24e52b6d48bef8f"
format_stamp: "Formatted at 2026-06-20 22:58:40 on dist-test-slave-f63b"
server_key: "5d896337cdf164319071c541e64c5210"
server_key_iv: "1a63c739039b80ba11704bc496853309"
server_key_version: "encryptionkey@0"
I20260620 22:58:40.512627 3016 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260620 22:58:40.516038 3031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:40.517328 3016 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260620 22:58:40.517508 3016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "809781b7e9134b45b24e52b6d48bef8f"
format_stamp: "Formatted at 2026-06-20 22:58:40 on dist-test-slave-f63b"
server_key: "5d896337cdf164319071c541e64c5210"
server_key_iv: "1a63c739039b80ba11704bc496853309"
server_key_version: "encryptionkey@0"
I20260620 22:58:40.517741 3016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:58:40.550441 3016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:58:40.554867 3016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:58:40.555166 3016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:58:40.574729 3016 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:46045
I20260620 22:58:40.574770 3082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:46045 every 8 connection(s)
I20260620 22:58:40.576107 3016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 22:58:40.579870 3083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:58:40.581611 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 3016
I20260620 22:58:40.581933 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 22:58:40.584321 2996 external_mini_cluster.cc:1428] Setting key 77a3491de7db4e1bba5bef6bcc66783a
I20260620 22:58:40.593654 3083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 809781b7e9134b45b24e52b6d48bef8f: Bootstrap starting.
I20260620 22:58:40.596732 3083 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 809781b7e9134b45b24e52b6d48bef8f: Neither blocks nor log segments found. Creating new log.
I20260620 22:58:40.597723 3083 log.cc:824] T 00000000000000000000000000000000 P 809781b7e9134b45b24e52b6d48bef8f: Log is configured to *not* fsync() on all Append() calls
I20260620 22:58:40.600713 3083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 809781b7e9134b45b24e52b6d48bef8f: No bootstrap required, opened a new log
I20260620 22:58:40.607933 3083 raft_consensus.cc:348] T 00000000000000000000000000000000 P 809781b7e9134b45b24e52b6d48bef8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "809781b7e9134b45b24e52b6d48bef8f" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 46045 } }
I20260620 22:58:40.608336 3083 raft_consensus.cc:374] T 00000000000000000000000000000000 P 809781b7e9134b45b24e52b6d48bef8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:58:40.608469 3083 raft_consensus.cc:729] T 00000000000000000000000000000000 P 809781b7e9134b45b24e52b6d48bef8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 809781b7e9134b45b24e52b6d48bef8f, State: Initialized, Role: FOLLOWER
I20260620 22:58:40.609619 3083 consensus_queue.cc:260] T 00000000000000000000000000000000 P 809781b7e9134b45b24e52b6d48bef8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "809781b7e9134b45b24e52b6d48bef8f" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 46045 } }
I20260620 22:58:40.609946 3083 raft_consensus.cc:388] T 00000000000000000000000000000000 P 809781b7e9134b45b24e52b6d48bef8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 22:58:40.610090 3083 raft_consensus.cc:482] T 00000000000000000000000000000000 P 809781b7e9134b45b24e52b6d48bef8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 22:58:40.610275 3083 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 809781b7e9134b45b24e52b6d48bef8f [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:58:40.613696 3083 raft_consensus.cc:504] T 00000000000000000000000000000000 P 809781b7e9134b45b24e52b6d48bef8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "809781b7e9134b45b24e52b6d48bef8f" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 46045 } }
I20260620 22:58:40.614292 3083 leader_election.cc:302] T 00000000000000000000000000000000 P 809781b7e9134b45b24e52b6d48bef8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 809781b7e9134b45b24e52b6d48bef8f; no voters:
Jun 20 22:58:40 dist-test-slave-f63b krb5kdc[3003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:58:40.614945 3083 leader_election.cc:288] T 00000000000000000000000000000000 P 809781b7e9134b45b24e52b6d48bef8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 22:58:40.615105 3088 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 809781b7e9134b45b24e52b6d48bef8f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:58:40.616675 3088 raft_consensus.cc:686] T 00000000000000000000000000000000 P 809781b7e9134b45b24e52b6d48bef8f [term 1 LEADER]: Becoming Leader. State: Replica: 809781b7e9134b45b24e52b6d48bef8f, State: Running, Role: LEADER
I20260620 22:58:40.617295 3083 sys_catalog.cc:564] T 00000000000000000000000000000000 P 809781b7e9134b45b24e52b6d48bef8f [sys.catalog]: configured and running, proceeding with master startup.
I20260620 22:58:40.617254 3088 consensus_queue.cc:237] T 00000000000000000000000000000000 P 809781b7e9134b45b24e52b6d48bef8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "809781b7e9134b45b24e52b6d48bef8f" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 46045 } }
I20260620 22:58:40.620828 3090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 809781b7e9134b45b24e52b6d48bef8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 809781b7e9134b45b24e52b6d48bef8f. Latest consensus state: current_term: 1 leader_uuid: "809781b7e9134b45b24e52b6d48bef8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "809781b7e9134b45b24e52b6d48bef8f" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 46045 } } }
I20260620 22:58:40.621235 3090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 809781b7e9134b45b24e52b6d48bef8f [sys.catalog]: This master's current role is: LEADER
I20260620 22:58:40.621850 3089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 809781b7e9134b45b24e52b6d48bef8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "809781b7e9134b45b24e52b6d48bef8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "809781b7e9134b45b24e52b6d48bef8f" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 46045 } } }
I20260620 22:58:40.622143 3089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 809781b7e9134b45b24e52b6d48bef8f [sys.catalog]: This master's current role is: LEADER
I20260620 22:58:40.625947 3095 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 22:58:40.628813 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:40.588911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38900 (local address 127.2.237.62:46045)
0620 22:58:40.591544 (+ 2633us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:40.591559 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:40.591615 (+ 56us) server_negotiation.cc:407] Connection header received
0620 22:58:40.600702 (+ 9087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:40.600724 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:40.601183 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:40.601588 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:40.604523 (+ 2935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:40.605852 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:40.607576 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:40.608364 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:40.608578 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:40.616564 (+ 7986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:40.616579 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:40.616600 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:40.616669 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:40.621891 (+ 5222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:40.623587 (+ 1696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:40.623600 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:40.623611 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:40.623865 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:40.624601 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:40.624609 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:40.624615 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:40.625544 (+ 929us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:40.625912 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:40.626855 (+ 943us) server_negotiation.cc:299] Negotiation successful
0620 22:58:40.627263 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2364,"thread_start_us":2129,"threads_started":1}
I20260620 22:58:40.632912 3095 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 22:58:40.642642 3095 catalog_manager.cc:1347] Generated new cluster ID: 12181d5f2e734bdbb3f279c72656cca9
I20260620 22:58:40.642765 3095 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 22:58:40.667883 3095 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 22:58:40.669375 3095 catalog_manager.cc:1504] Loading token signing keys...
I20260620 22:58:40.682745 3095 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 809781b7e9134b45b24e52b6d48bef8f: Generated new TSK 0
I20260620 22:58:40.683874 3095 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.237.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
I20260620 22:58:40.747171 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:0
--local_ip_for_outbound_sockets=127.2.237.1
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:46045
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:41123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:58:41.095642 3109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:58:41.095919 3109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:58:41.096014 3109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:58:41.103659 3109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:58:41.103768 3109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:58:41.103883 3109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 22:58:41.113818 3109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:41123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:46045
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:58:41.115146 3109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:58:41.116377 3109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:58:41.130761 3117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:41.130739 3114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:41.131536 3115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:58:41.132514 3109 server_base.cc:1034] running on GCE node
I20260620 22:58:41.133245 3109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:58:41.134248 3109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:58:41.135439 3109 hybrid_clock.cc:648] HybridClock initialized: now 1781996321135393 us; error 41 us; skew 500 ppm
I20260620 22:58:41.135771 3109 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:58:41 dist-test-slave-f63b krb5kdc[3003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996321, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:58:41.139029 3109 init.cc:377] Logged in from keytab as kudu/127.2.237.1@KRBTEST.COM (short username kudu)
I20260620 22:58:41.140548 3109 webserver.cc:460] Webserver started at http://127.2.237.1:45243/ using document root <none> and password file <none>
I20260620 22:58:41.141252 3109 fs_manager.cc:359] Metadata directory not provided
I20260620 22:58:41.141398 3109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:58:41.141716 3109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:58:41.145080 3109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-0/data/instance:
uuid: "66a64ba0232a4366b4c790cd95821e71"
format_stamp: "Formatted at 2026-06-20 22:58:41 on dist-test-slave-f63b"
server_key: "5bf6582dece0b7e62c65d6edbec0d90a"
server_key_iv: "5b409b3036ccfff8f801847da91d9f0f"
server_key_version: "encryptionkey@0"
I20260620 22:58:41.145854 3109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance:
uuid: "66a64ba0232a4366b4c790cd95821e71"
format_stamp: "Formatted at 2026-06-20 22:58:41 on dist-test-slave-f63b"
server_key: "5bf6582dece0b7e62c65d6edbec0d90a"
server_key_iv: "5b409b3036ccfff8f801847da91d9f0f"
server_key_version: "encryptionkey@0"
I20260620 22:58:41.151539 3109 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260620 22:58:41.155169 3124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:41.156599 3109 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260620 22:58:41.156780 3109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "66a64ba0232a4366b4c790cd95821e71"
format_stamp: "Formatted at 2026-06-20 22:58:41 on dist-test-slave-f63b"
server_key: "5bf6582dece0b7e62c65d6edbec0d90a"
server_key_iv: "5b409b3036ccfff8f801847da91d9f0f"
server_key_version: "encryptionkey@0"
I20260620 22:58:41.156982 3109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:58:41.188788 3109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:58:41.193332 3109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:58:41.193629 3109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:58:41.194717 3109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:58:41.196087 3109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:58:41.196189 3109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:41.196358 3109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:58:41.196439 3109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:41.233064 3109 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:40853
I20260620 22:58:41.233155 3236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:40853 every 8 connection(s)
I20260620 22:58:41.234382 3109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 22:58:41.237394 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 3109
I20260620 22:58:41.237618 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance
I20260620 22:58:41.238447 2996 external_mini_cluster.cc:1428] Setting key 71dc7207c6ca9dcc064ffcc794eaf320
Jun 20 22:58:41 dist-test-slave-f63b krb5kdc[3003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996321, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.237.2@KRBTEST.COM; defaulting to no policy
I20260620 22:58:41.255055 3239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:41.237036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:51635 (local address 127.2.237.62:46045)
0620 22:58:41.237538 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:41.237545 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:41.238627 (+ 1082us) server_negotiation.cc:407] Connection header received
0620 22:58:41.241321 (+ 2694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:41.241326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:41.241402 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:41.241526 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:41.243082 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:41.243798 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:41.245327 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:41.245723 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:41.245870 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:41.249290 (+ 3420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:41.249296 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:41.249301 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:41.249342 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:41.251947 (+ 2605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:41.252682 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:41.252687 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:41.252691 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:41.252810 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:41.253266 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:41.253271 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:41.253275 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:41.253585 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:41.253740 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:41.254563 (+ 823us) server_negotiation.cc:299] Negotiation successful
0620 22:58:41.254784 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":242,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.2@KRBTEST.COM" created.
I20260620 22:58:41.256120 3240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:41.237677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:46045 (local address 127.2.237.1:51635)
0620 22:58:41.238435 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:41.238477 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:41.241065 (+ 2588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:41.241762 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:41.241771 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:41.242275 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:58:41.242924 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:41.242942 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:41.243974 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:41.243978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:41.245189 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:41.245200 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:41.245946 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:41.245954 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:41.246188 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:41.246918 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:58:41.246939 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:58:41.249075 (+ 2136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:58:41.252094 (+ 3019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:41.252102 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:41.252121 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:41.252554 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:41.252949 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:41.252956 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:41.252959 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:41.253149 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:41.253716 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:58:41.253724 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:58:41.254078 (+ 354us) client_negotiation.cc:769] Sending connection context
0620 22:58:41.254367 (+ 289us) client_negotiation.cc:240] Negotiation successful
0620 22:58:41.254875 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":314,"threads_started":1}
I20260620 22:58:41.257797 3237 heartbeater.cc:347] Connected to a master server at 127.2.237.62:46045
I20260620 22:58:41.258142 3237 heartbeater.cc:464] Registering TS with master...
I20260620 22:58:41.259295 3237 heartbeater.cc:511] Master 127.2.237.62:46045 requested a full tablet report, sending...
I20260620 22:58:41.262112 3048 ts_manager.cc:194] Registered new tserver with Master: 66a64ba0232a4366b4c790cd95821e71 (127.2.237.1:40853)
I20260620 22:58:41.263687 3048 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.1@KRBTEST.COM'} at 127.2.237.1:51635
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
I20260620 22:58:41.281647 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:0
--local_ip_for_outbound_sockets=127.2.237.2
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:46045
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:41123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:58:41.610014 3243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:58:41.610349 3243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:58:41.610512 3243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:58:41.617837 3243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:58:41.617976 3243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:58:41.618160 3243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 22:58:41.628058 3243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:41123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:46045
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:58:41.629441 3243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:58:41.631309 3243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:58:41.644716 3248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:41.644701 3249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:41.644708 3251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:58:41.645475 3243 server_base.cc:1034] running on GCE node
I20260620 22:58:41.646045 3243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:58:41.647060 3243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:58:41.648258 3243 hybrid_clock.cc:648] HybridClock initialized: now 1781996321648223 us; error 38 us; skew 500 ppm
I20260620 22:58:41.648546 3243 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:58:41 dist-test-slave-f63b krb5kdc[3003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996321, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:58:41.651671 3243 init.cc:377] Logged in from keytab as kudu/127.2.237.2@KRBTEST.COM (short username kudu)
I20260620 22:58:41.653148 3243 webserver.cc:460] Webserver started at http://127.2.237.2:46023/ using document root <none> and password file <none>
I20260620 22:58:41.653836 3243 fs_manager.cc:359] Metadata directory not provided
I20260620 22:58:41.653967 3243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:58:41.654287 3243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:58:41.657367 3243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-1/data/instance:
uuid: "12be28fd5c95489bb786e649c14f4cae"
format_stamp: "Formatted at 2026-06-20 22:58:41 on dist-test-slave-f63b"
server_key: "81b480c219624b86493fd93693341007"
server_key_iv: "0357580668dae31ea1240327a7f37ddf"
server_key_version: "encryptionkey@0"
I20260620 22:58:41.658128 3243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance:
uuid: "12be28fd5c95489bb786e649c14f4cae"
format_stamp: "Formatted at 2026-06-20 22:58:41 on dist-test-slave-f63b"
server_key: "81b480c219624b86493fd93693341007"
server_key_iv: "0357580668dae31ea1240327a7f37ddf"
server_key_version: "encryptionkey@0"
I20260620 22:58:41.663194 3243 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260620 22:58:41.666486 3258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:41.667969 3243 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 22:58:41.668140 3243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "12be28fd5c95489bb786e649c14f4cae"
format_stamp: "Formatted at 2026-06-20 22:58:41 on dist-test-slave-f63b"
server_key: "81b480c219624b86493fd93693341007"
server_key_iv: "0357580668dae31ea1240327a7f37ddf"
server_key_version: "encryptionkey@0"
I20260620 22:58:41.668337 3243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:58:41.681965 3243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:58:41.685824 3243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:58:41.686123 3243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:58:41.687093 3243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:58:41.688407 3243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:58:41.688519 3243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:41.688678 3243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:58:41.688771 3243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:41.724552 3243 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:44181
I20260620 22:58:41.724586 3370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:44181 every 8 connection(s)
I20260620 22:58:41.726114 3243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 22:58:41.728291 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 3243
I20260620 22:58:41.728490 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance
I20260620 22:58:41.728972 2996 external_mini_cluster.cc:1428] Setting key ab9eaae8334861ac6315f31cb91e3a2d
Jun 20 22:58:41 dist-test-slave-f63b krb5kdc[3003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996321, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.237.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.3@KRBTEST.COM" created.
I20260620 22:58:41.749732 3239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:41.729397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:39805 (local address 127.2.237.62:46045)
0620 22:58:41.729623 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:41.729628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:41.731174 (+ 1546us) server_negotiation.cc:407] Connection header received
0620 22:58:41.734059 (+ 2885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:41.734063 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:41.734123 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:41.734201 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:41.736659 (+ 2458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:41.737829 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:41.739468 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:41.740243 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:41.740382 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:41.743892 (+ 3510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:41.743897 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:41.743901 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:41.743933 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:41.746439 (+ 2506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:41.747178 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:41.747187 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:41.747189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:41.747308 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:41.747880 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:41.747885 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:41.747889 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:41.748165 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:41.748374 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:41.749146 (+ 772us) server_negotiation.cc:299] Negotiation successful
0620 22:58:41.749366 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260620 22:58:41.750780 3373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:41.730077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:46045 (local address 127.2.237.2:39805)
0620 22:58:41.731011 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:41.731060 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:41.733756 (+ 2696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:41.734913 (+ 1157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:41.734927 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:41.735672 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:58:41.736494 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:41.736514 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:41.737972 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:41.737976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:41.739328 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:41.739339 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:41.740439 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:41.740443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:41.740634 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:41.741359 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:58:41.741380 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:58:41.743650 (+ 2270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:58:41.746620 (+ 2970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:41.746629 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:41.746645 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:41.747050 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:41.747575 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:41.747578 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:41.747580 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:41.747757 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:41.748313 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:58:41.748320 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:58:41.748695 (+ 375us) client_negotiation.cc:769] Sending connection context
0620 22:58:41.749040 (+ 345us) client_negotiation.cc:240] Negotiation successful
0620 22:58:41.749562 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":398,"threads_started":1}
I20260620 22:58:41.752203 3371 heartbeater.cc:347] Connected to a master server at 127.2.237.62:46045
I20260620 22:58:41.752533 3371 heartbeater.cc:464] Registering TS with master...
I20260620 22:58:41.753402 3371 heartbeater.cc:511] Master 127.2.237.62:46045 requested a full tablet report, sending...
I20260620 22:58:41.754963 3048 ts_manager.cc:194] Registered new tserver with Master: 12be28fd5c95489bb786e649c14f4cae (127.2.237.2:44181)
I20260620 22:58:41.756266 3048 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.2@KRBTEST.COM'} at 127.2.237.2:39805
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
I20260620 22:58:41.771768 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:0
--local_ip_for_outbound_sockets=127.2.237.3
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:46045
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:41123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:58:42.102157 3376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:58:42.102428 3376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:58:42.102625 3376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:58:42.109865 3376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:58:42.109967 3376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:58:42.110179 3376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 22:58:42.120339 3376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:41123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:46045
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:58:42.121711 3376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:58:42.123106 3376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:58:42.137415 3384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:42.137439 3382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:42.137549 3381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:58:42.139325 3376 server_base.cc:1034] running on GCE node
I20260620 22:58:42.139974 3376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:58:42.140935 3376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:58:42.142179 3376 hybrid_clock.cc:648] HybridClock initialized: now 1781996322142131 us; error 67 us; skew 500 ppm
I20260620 22:58:42.142498 3376 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:58:42 dist-test-slave-f63b krb5kdc[3003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996322, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:58:42.145615 3376 init.cc:377] Logged in from keytab as kudu/127.2.237.3@KRBTEST.COM (short username kudu)
I20260620 22:58:42.147097 3376 webserver.cc:460] Webserver started at http://127.2.237.3:33681/ using document root <none> and password file <none>
I20260620 22:58:42.147789 3376 fs_manager.cc:359] Metadata directory not provided
I20260620 22:58:42.147936 3376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:58:42.148252 3376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:58:42.151394 3376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-2/data/instance:
uuid: "2154ab6dcfbe4aa6a52e4f22e82200da"
format_stamp: "Formatted at 2026-06-20 22:58:42 on dist-test-slave-f63b"
server_key: "6090eeb56486d638652b3ee894356f3d"
server_key_iv: "4b0add8cd6daae154f052cfbe7a6a22c"
server_key_version: "encryptionkey@0"
I20260620 22:58:42.152174 3376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance:
uuid: "2154ab6dcfbe4aa6a52e4f22e82200da"
format_stamp: "Formatted at 2026-06-20 22:58:42 on dist-test-slave-f63b"
server_key: "6090eeb56486d638652b3ee894356f3d"
server_key_iv: "4b0add8cd6daae154f052cfbe7a6a22c"
server_key_version: "encryptionkey@0"
I20260620 22:58:42.157510 3376 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260620 22:58:42.161010 3391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:42.162423 3376 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260620 22:58:42.163028 3376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "2154ab6dcfbe4aa6a52e4f22e82200da"
format_stamp: "Formatted at 2026-06-20 22:58:42 on dist-test-slave-f63b"
server_key: "6090eeb56486d638652b3ee894356f3d"
server_key_iv: "4b0add8cd6daae154f052cfbe7a6a22c"
server_key_version: "encryptionkey@0"
I20260620 22:58:42.163262 3376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:58:42.191344 3376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:58:42.195192 3376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:58:42.195492 3376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:58:42.196430 3376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:58:42.197769 3376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:58:42.197870 3376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:42.198022 3376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:58:42.198093 3376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:42.234078 3376 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:39747
I20260620 22:58:42.234100 3503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:39747 every 8 connection(s)
I20260620 22:58:42.235414 3376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 22:58:42.238343 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 3376
I20260620 22:58:42.238559 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance
I20260620 22:58:42.239066 2996 external_mini_cluster.cc:1428] Setting key 4abac49f4eacfc124f0114c2be1f4517
Jun 20 22:58:42 dist-test-slave-f63b krb5kdc[3003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996322, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:58:42.256189 3239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:42.238052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:44815 (local address 127.2.237.62:46045)
0620 22:58:42.238281 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:42.238289 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:42.239675 (+ 1386us) server_negotiation.cc:407] Connection header received
0620 22:58:42.242037 (+ 2362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:42.242042 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:42.242125 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:42.242249 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:42.243805 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:42.244523 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:42.245952 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:42.246333 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:42.246419 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:42.250040 (+ 3621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:42.250045 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:42.250048 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:42.250080 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:42.252833 (+ 2753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:42.253624 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:42.253629 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:42.253631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:42.253759 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:42.254246 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:42.254249 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:42.254251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:42.254676 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:42.254826 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:42.255601 (+ 775us) server_negotiation.cc:299] Negotiation successful
0620 22:58:42.255821 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260620 22:58:42.257241 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:42.238617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:46045 (local address 127.2.237.3:44815)
0620 22:58:42.239487 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:42.239535 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:42.241793 (+ 2258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:42.242458 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:42.242466 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:42.243142 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:58:42.243663 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:42.243679 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:42.244650 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:42.244654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:42.245840 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:42.245850 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:42.246505 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:42.246509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:42.246781 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:42.247378 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:58:42.247402 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:58:42.249833 (+ 2431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:58:42.252951 (+ 3118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:42.252957 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:42.252969 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:42.253504 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:42.253888 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:42.253894 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:42.253900 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:42.254133 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:42.254796 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:58:42.254804 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:58:42.255225 (+ 421us) client_negotiation.cc:769] Sending connection context
0620 22:58:42.255514 (+ 289us) client_negotiation.cc:240] Negotiation successful
0620 22:58:42.256040 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":294,"threads_started":1}
I20260620 22:58:42.258613 3504 heartbeater.cc:347] Connected to a master server at 127.2.237.62:46045
I20260620 22:58:42.258919 3504 heartbeater.cc:464] Registering TS with master...
I20260620 22:58:42.259622 3504 heartbeater.cc:511] Master 127.2.237.62:46045 requested a full tablet report, sending...
I20260620 22:58:42.261008 3048 ts_manager.cc:194] Registered new tserver with Master: 2154ab6dcfbe4aa6a52e4f22e82200da (127.2.237.3:39747)
I20260620 22:58:42.261802 3048 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.3@KRBTEST.COM'} at 127.2.237.3:44815
I20260620 22:58:42.267457 3237 heartbeater.cc:503] Master 127.2.237.62:46045 was elected leader, sending a full tablet report...
I20260620 22:58:42.267668 2996 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 22:58:42 dist-test-slave-f63b krb5kdc[3003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996322, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 22:58:42 dist-test-slave-f63b krb5kdc[3003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996322, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.237.1@KRBTEST.COM
I20260620 22:58:42.312690 3511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:42.297161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56682 (local address 127.2.237.1:40853)
0620 22:58:42.297826 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:42.297834 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:42.297881 (+ 47us) server_negotiation.cc:407] Connection header received
0620 22:58:42.298031 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:42.298038 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:42.298267 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:42.298396 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:42.300297 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:42.301364 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:42.302547 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:42.303237 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:42.303346 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:42.307145 (+ 3799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:42.307156 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:42.307170 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:42.307206 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:42.310214 (+ 3008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:42.310890 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:42.310899 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:42.310901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:42.311022 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:42.311340 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:42.311343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:42.311345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:42.311679 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:42.311894 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:42.312117 (+ 223us) server_negotiation.cc:299] Negotiation successful
0620 22:58:42.312339 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":323,"threads_started":1}
W20260620 22:58:42.313879 3152 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56682
Jun 20 22:58:42 dist-test-slave-f63b krb5kdc[3003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 22:58:42 dist-test-slave-f63b krb5kdc[3003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.1@KRBTEST.COM
I20260620 22:58:42.356616 3511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:42.343318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56690 (local address 127.2.237.1:40853)
0620 22:58:42.343552 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:42.343560 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:42.343764 (+ 204us) server_negotiation.cc:407] Connection header received
0620 22:58:42.343992 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:42.343997 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:42.344113 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:42.344225 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:42.345597 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:42.346316 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:42.347607 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:42.348012 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:42.348141 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:42.351985 (+ 3844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:42.351990 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:42.351993 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:42.352027 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:42.354355 (+ 2328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:42.355049 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:42.355053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:42.355055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:42.355171 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:42.355544 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:42.355547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:42.355549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:42.355778 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:42.355943 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:42.356219 (+ 276us) server_negotiation.cc:299] Negotiation successful
0620 22:58:42.356375 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260620 22:58:42.358222 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 3109
I20260620 22:58:42.371407 3511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:42.359272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56700 (local address 127.2.237.1:40853)
0620 22:58:42.359522 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:42.359527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:42.359807 (+ 280us) server_negotiation.cc:407] Connection header received
0620 22:58:42.360023 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:42.360028 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:42.360091 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:42.360192 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:42.361394 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:42.362069 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:42.363378 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:42.363793 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:42.363937 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:42.366230 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:42.366235 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:42.366238 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:42.366267 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:42.369126 (+ 2859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:42.369708 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:42.369711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:42.369713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:42.369829 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:42.370190 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:42.370193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:42.370195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:42.370428 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:42.370665 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:42.370975 (+ 310us) server_negotiation.cc:299] Negotiation successful
0620 22:58:42.371132 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260620 22:58:42.372124 3232 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:58:42.455708 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 3109
I20260620 22:58:42.472504 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 3243
Jun 20 22:58:42 dist-test-slave-f63b krb5kdc[3003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.2@KRBTEST.COM
I20260620 22:58:42.488293 3518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:42.473328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37514 (local address 127.2.237.2:44181)
0620 22:58:42.473935 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:42.473942 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:42.473969 (+ 27us) server_negotiation.cc:407] Connection header received
0620 22:58:42.474029 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:42.474035 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:42.474246 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:42.474378 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:42.475555 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:42.476482 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:42.477633 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:42.478174 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:42.478312 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:42.482252 (+ 3940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:42.482264 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:42.482290 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:42.482326 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:42.485383 (+ 3057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:42.486013 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:42.486021 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:42.486023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:42.486152 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:42.486521 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:42.486525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:42.486527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:42.487151 (+ 624us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:42.487399 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:42.487647 (+ 248us) server_negotiation.cc:299] Negotiation successful
0620 22:58:42.487917 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":289,"threads_started":1}
I20260620 22:58:42.489632 3366 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:58:42.571856 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 3243
I20260620 22:58:42.593995 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 3376
Jun 20 22:58:42 dist-test-slave-f63b krb5kdc[3003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.3@KRBTEST.COM
I20260620 22:58:42.610389 3520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:42.595124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44576 (local address 127.2.237.3:39747)
0620 22:58:42.595731 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:42.595739 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:42.595768 (+ 29us) server_negotiation.cc:407] Connection header received
0620 22:58:42.595833 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:42.595840 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:42.596083 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:42.596214 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:42.597392 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:42.598363 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:42.599703 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:42.600250 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:42.600346 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:42.604704 (+ 4358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:42.604714 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:42.604729 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:42.604762 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:42.607619 (+ 2857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:42.608309 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:42.608316 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:42.608318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:42.608441 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:42.608820 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:42.608829 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:42.608831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:42.609170 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:42.609478 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:42.609790 (+ 312us) server_negotiation.cc:299] Negotiation successful
0620 22:58:42.610061 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":328,"threads_started":1}
I20260620 22:58:42.611768 3499 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:58:42.695771 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 3376
I20260620 22:58:42.715191 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 3016
I20260620 22:58:42.716341 3078 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:58:42.819489 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 3016
2026-06-20T22:58:42Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3977 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 22:58:42 dist-test-slave-f63b krb5kdc[3525](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 22:58:42 dist-test-slave-f63b krb5kdc[3525](info): set up 2 sockets
Jun 20 22:58:42 dist-test-slave-f63b krb5kdc[3525](info): commencing operation
krb5kdc: starting...
W20260620 22:58:43.895187 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.000s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 22:58:44 dist-test-slave-f63b krb5kdc[3525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T22:58:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T22:58:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
I20260620 22:58:44.074268 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:38775
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:43795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:38775
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:58:44.397926 3538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:58:44.398209 3538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:58:44.398298 3538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:58:44.405577 3538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 22:58:44.405689 3538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:58:44.405737 3538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:58:44.405790 3538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 22:58:44.405840 3538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 22:58:44.415740 3538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:43795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:38775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:38775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:58:44.417016 3538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:58:44.418308 3538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:58:44.429961 3543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:44.430014 3544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:44.430012 3546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:58:44.431322 3538 server_base.cc:1034] running on GCE node
I20260620 22:58:44.432111 3538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:58:44.433459 3538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:58:44.434681 3538 hybrid_clock.cc:648] HybridClock initialized: now 1781996324434640 us; error 39 us; skew 500 ppm
I20260620 22:58:44.435075 3538 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:58:44 dist-test-slave-f63b krb5kdc[3525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996324, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:58:44.441617 3538 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 22:58:44.443352 3538 webserver.cc:460] Webserver started at http://127.2.237.62:44793/ using document root <none> and password file <none>
I20260620 22:58:44.444074 3538 fs_manager.cc:359] Metadata directory not provided
I20260620 22:58:44.444177 3538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:58:44.444526 3538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:58:44.447563 3538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "c3d182fb2800419d917a1d60208653a5"
format_stamp: "Formatted at 2026-06-20 22:58:44 on dist-test-slave-f63b"
server_key: "93b87e9e8e4f23a2714e69ec3c7c1c62"
server_key_iv: "3fea8c27b7fd66957e0aba2874e5ca66"
server_key_version: "encryptionkey@0"
I20260620 22:58:44.448316 3538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "c3d182fb2800419d917a1d60208653a5"
format_stamp: "Formatted at 2026-06-20 22:58:44 on dist-test-slave-f63b"
server_key: "93b87e9e8e4f23a2714e69ec3c7c1c62"
server_key_iv: "3fea8c27b7fd66957e0aba2874e5ca66"
server_key_version: "encryptionkey@0"
I20260620 22:58:44.453472 3538 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260620 22:58:44.456812 3553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:44.458144 3538 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 22:58:44.458328 3538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "c3d182fb2800419d917a1d60208653a5"
format_stamp: "Formatted at 2026-06-20 22:58:44 on dist-test-slave-f63b"
server_key: "93b87e9e8e4f23a2714e69ec3c7c1c62"
server_key_iv: "3fea8c27b7fd66957e0aba2874e5ca66"
server_key_version: "encryptionkey@0"
I20260620 22:58:44.458525 3538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:58:44.511030 3538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:58:44.515245 3538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:58:44.515566 3538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:58:44.533722 3538 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:38775
I20260620 22:58:44.533730 3604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:38775 every 8 connection(s)
I20260620 22:58:44.535187 3538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 22:58:44.538887 3605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:58:44.541225 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 3538
I20260620 22:58:44.541443 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 22:58:44.542068 2996 external_mini_cluster.cc:1428] Setting key b99254b4a46509885b6443c616563648
I20260620 22:58:44.550717 3605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3d182fb2800419d917a1d60208653a5: Bootstrap starting.
I20260620 22:58:44.554385 3605 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3d182fb2800419d917a1d60208653a5: Neither blocks nor log segments found. Creating new log.
I20260620 22:58:44.555716 3605 log.cc:824] T 00000000000000000000000000000000 P c3d182fb2800419d917a1d60208653a5: Log is configured to *not* fsync() on all Append() calls
Jun 20 22:58:44 dist-test-slave-f63b krb5kdc[3525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:58:44.558753 3605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3d182fb2800419d917a1d60208653a5: No bootstrap required, opened a new log
I20260620 22:58:44.565999 3608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:44.544011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36670 (local address 127.2.237.62:38775)
0620 22:58:44.544781 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:44.544792 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:44.544832 (+ 40us) server_negotiation.cc:407] Connection header received
0620 22:58:44.546496 (+ 1664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:44.546544 (+ 48us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:44.547031 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:44.547350 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:44.549138 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:44.550400 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:44.552129 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:44.552807 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:44.552932 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:44.557579 (+ 4647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:44.557587 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:44.557603 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:44.557640 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:44.561019 (+ 3379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:44.561767 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:44.561777 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:44.561789 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:44.562022 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:44.562447 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:44.562453 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:44.562456 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:44.563312 (+ 856us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:44.563653 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:44.564088 (+ 435us) server_negotiation.cc:299] Negotiation successful
0620 22:58:44.564564 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":361,"threads_started":1}
I20260620 22:58:44.568347 3605 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3d182fb2800419d917a1d60208653a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3d182fb2800419d917a1d60208653a5" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 38775 } }
I20260620 22:58:44.568746 3605 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3d182fb2800419d917a1d60208653a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:58:44.568864 3605 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3d182fb2800419d917a1d60208653a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3d182fb2800419d917a1d60208653a5, State: Initialized, Role: FOLLOWER
I20260620 22:58:44.569407 3605 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3d182fb2800419d917a1d60208653a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3d182fb2800419d917a1d60208653a5" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 38775 } }
I20260620 22:58:44.569609 3605 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3d182fb2800419d917a1d60208653a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 22:58:44.569700 3605 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3d182fb2800419d917a1d60208653a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 22:58:44.569818 3605 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3d182fb2800419d917a1d60208653a5 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:58:44.571938 3605 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3d182fb2800419d917a1d60208653a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3d182fb2800419d917a1d60208653a5" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 38775 } }
I20260620 22:58:44.572350 3605 leader_election.cc:302] T 00000000000000000000000000000000 P c3d182fb2800419d917a1d60208653a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3d182fb2800419d917a1d60208653a5; no voters:
I20260620 22:58:44.572865 3605 leader_election.cc:288] T 00000000000000000000000000000000 P c3d182fb2800419d917a1d60208653a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 22:58:44.573033 3610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3d182fb2800419d917a1d60208653a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:58:44.573678 3610 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3d182fb2800419d917a1d60208653a5 [term 1 LEADER]: Becoming Leader. State: Replica: c3d182fb2800419d917a1d60208653a5, State: Running, Role: LEADER
I20260620 22:58:44.574170 3610 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3d182fb2800419d917a1d60208653a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3d182fb2800419d917a1d60208653a5" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 38775 } }
I20260620 22:58:44.575024 3605 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3d182fb2800419d917a1d60208653a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 22:58:44.577390 3612 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3d182fb2800419d917a1d60208653a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3d182fb2800419d917a1d60208653a5. Latest consensus state: current_term: 1 leader_uuid: "c3d182fb2800419d917a1d60208653a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3d182fb2800419d917a1d60208653a5" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 38775 } } }
I20260620 22:58:44.577363 3611 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3d182fb2800419d917a1d60208653a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3d182fb2800419d917a1d60208653a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3d182fb2800419d917a1d60208653a5" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 38775 } } }
I20260620 22:58:44.577811 3612 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3d182fb2800419d917a1d60208653a5 [sys.catalog]: This master's current role is: LEADER
I20260620 22:58:44.577849 3611 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3d182fb2800419d917a1d60208653a5 [sys.catalog]: This master's current role is: LEADER
I20260620 22:58:44.582710 3615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 22:58:44.587795 3615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 22:58:44.596663 3615 catalog_manager.cc:1347] Generated new cluster ID: 1b51d2d5d8e148e3951584968efcda86
I20260620 22:58:44.596801 3615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 22:58:44.616989 3615 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 22:58:44.618400 3615 catalog_manager.cc:1504] Loading token signing keys...
I20260620 22:58:44.630383 3615 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3d182fb2800419d917a1d60208653a5: Generated new TSK 0
I20260620 22:58:44.631269 3615 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.237.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
I20260620 22:58:44.674141 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:0
--local_ip_for_outbound_sockets=127.2.237.1
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:38775
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:43795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:58:45.001726 3631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:58:45.002040 3631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:58:45.002138 3631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:58:45.009264 3631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:58:45.009377 3631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:58:45.009493 3631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 22:58:45.019281 3631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:43795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:38775
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:58:45.020516 3631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:58:45.021867 3631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:58:45.035112 3636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:45.035048 3637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:45.035089 3639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:58:45.036229 3631 server_base.cc:1034] running on GCE node
I20260620 22:58:45.036813 3631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:58:45.037730 3631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:58:45.038957 3631 hybrid_clock.cc:648] HybridClock initialized: now 1781996325038917 us; error 35 us; skew 500 ppm
I20260620 22:58:45.039283 3631 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:58:45 dist-test-slave-f63b krb5kdc[3525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996325, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:58:45.042366 3631 init.cc:377] Logged in from keytab as kudu/127.2.237.1@KRBTEST.COM (short username kudu)
I20260620 22:58:45.043880 3631 webserver.cc:460] Webserver started at http://127.2.237.1:38585/ using document root <none> and password file <none>
I20260620 22:58:45.044605 3631 fs_manager.cc:359] Metadata directory not provided
I20260620 22:58:45.044701 3631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:58:45.044947 3631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:58:45.047819 3631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-0/data/instance:
uuid: "b0187ee975cc4a5d8bf65fd98fb29ad6"
format_stamp: "Formatted at 2026-06-20 22:58:45 on dist-test-slave-f63b"
server_key: "32b3b2ecbcfac6bc066aaedb4c669843"
server_key_iv: "27695684ed1dccb848f3723745c8700b"
server_key_version: "encryptionkey@0"
I20260620 22:58:45.048590 3631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance:
uuid: "b0187ee975cc4a5d8bf65fd98fb29ad6"
format_stamp: "Formatted at 2026-06-20 22:58:45 on dist-test-slave-f63b"
server_key: "32b3b2ecbcfac6bc066aaedb4c669843"
server_key_iv: "27695684ed1dccb848f3723745c8700b"
server_key_version: "encryptionkey@0"
I20260620 22:58:45.053643 3631 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260620 22:58:45.057031 3646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:45.058382 3631 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 22:58:45.058617 3631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "b0187ee975cc4a5d8bf65fd98fb29ad6"
format_stamp: "Formatted at 2026-06-20 22:58:45 on dist-test-slave-f63b"
server_key: "32b3b2ecbcfac6bc066aaedb4c669843"
server_key_iv: "27695684ed1dccb848f3723745c8700b"
server_key_version: "encryptionkey@0"
I20260620 22:58:45.058830 3631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:58:45.083371 3631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:58:45.087783 3631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:58:45.088094 3631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:58:45.089083 3631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:58:45.090382 3631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:58:45.090466 3631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:45.090581 3631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:58:45.090659 3631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260620 22:58:45.124631 3631 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:37541
I20260620 22:58:45.124716 3758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:37541 every 8 connection(s)
I20260620 22:58:45.125967 3631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 22:58:45.133189 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 3631
I20260620 22:58:45.133400 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance
I20260620 22:58:45.134191 2996 external_mini_cluster.cc:1428] Setting key 189998c696d0ec962c4084f1664cb269
Jun 20 22:58:45 dist-test-slave-f63b krb5kdc[3525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996325, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:58:45.148203 3761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:45.128582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:32927 (local address 127.2.237.62:38775)
0620 22:58:45.129051 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:45.129058 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:45.130078 (+ 1020us) server_negotiation.cc:407] Connection header received
0620 22:58:45.131946 (+ 1868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:45.131950 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:45.132015 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:45.132111 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:45.133690 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:45.134786 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:45.136785 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:45.137492 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:45.137630 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:45.142097 (+ 4467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:45.142104 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:45.142110 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:45.142153 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:45.144865 (+ 2712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:45.145784 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:45.145787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:45.145790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:45.145898 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:45.146363 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:45.146366 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:45.146368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:45.146697 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:45.146805 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:45.147694 (+ 889us) server_negotiation.cc:299] Negotiation successful
0620 22:58:45.147849 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":227,"threads_started":1}
I20260620 22:58:45.149842 3762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:45.129193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:38775 (local address 127.2.237.1:32927)
0620 22:58:45.129905 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:45.129949 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:45.131695 (+ 1746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:45.132329 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:45.132336 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:45.132855 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:58:45.133517 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:45.133536 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:45.134942 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:45.134947 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:45.136650 (+ 1703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:45.136661 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:45.137701 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:45.137706 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:45.138000 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:45.138858 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:58:45.138886 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:58:45.141865 (+ 2979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:58:45.145008 (+ 3143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:45.145021 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:45.145042 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:45.145668 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:45.146000 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:45.146008 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:45.146014 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:45.146256 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:45.146817 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:58:45.146828 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:58:45.147292 (+ 464us) client_negotiation.cc:769] Sending connection context
0620 22:58:45.147617 (+ 325us) client_negotiation.cc:240] Negotiation successful
0620 22:58:45.148206 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":251,"threads_started":1}
I20260620 22:58:45.151503 3759 heartbeater.cc:347] Connected to a master server at 127.2.237.62:38775
I20260620 22:58:45.151829 3759 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.2.237.2@KRBTEST.COM; defaulting to no policy
I20260620 22:58:45.152588 3759 heartbeater.cc:511] Master 127.2.237.62:38775 requested a full tablet report, sending...
I20260620 22:58:45.154805 3570 ts_manager.cc:194] Registered new tserver with Master: b0187ee975cc4a5d8bf65fd98fb29ad6 (127.2.237.1:37541)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.2@KRBTEST.COM" created.
I20260620 22:58:45.156950 3570 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.1@KRBTEST.COM'} at 127.2.237.1:32927
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
I20260620 22:58:45.178045 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:0
--local_ip_for_outbound_sockets=127.2.237.2
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:38775
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:43795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:58:45.492882 3765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:58:45.493198 3765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:58:45.493356 3765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:58:45.500586 3765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:58:45.500739 3765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:58:45.500957 3765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 22:58:45.510643 3765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:43795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:38775
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:58:45.511996 3765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:58:45.513374 3765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:58:45.526412 3770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:45.526492 3773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:45.526458 3771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:58:45.527325 3765 server_base.cc:1034] running on GCE node
I20260620 22:58:45.527918 3765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:58:45.528818 3765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:58:45.530042 3765 hybrid_clock.cc:648] HybridClock initialized: now 1781996325530000 us; error 59 us; skew 500 ppm
I20260620 22:58:45.530360 3765 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:58:45 dist-test-slave-f63b krb5kdc[3525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996325, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:58:45.533501 3765 init.cc:377] Logged in from keytab as kudu/127.2.237.2@KRBTEST.COM (short username kudu)
I20260620 22:58:45.535197 3765 webserver.cc:460] Webserver started at http://127.2.237.2:40723/ using document root <none> and password file <none>
I20260620 22:58:45.535853 3765 fs_manager.cc:359] Metadata directory not provided
I20260620 22:58:45.535975 3765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:58:45.536298 3765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:58:45.539147 3765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-1/data/instance:
uuid: "1f15c8a4b0cb417fb4a84764af301e02"
format_stamp: "Formatted at 2026-06-20 22:58:45 on dist-test-slave-f63b"
server_key: "e4a98835d7eadb5a7d368dfc05d7781b"
server_key_iv: "a28e169ee71ea1b01f5ceeba8854c1d6"
server_key_version: "encryptionkey@0"
I20260620 22:58:45.539891 3765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance:
uuid: "1f15c8a4b0cb417fb4a84764af301e02"
format_stamp: "Formatted at 2026-06-20 22:58:45 on dist-test-slave-f63b"
server_key: "e4a98835d7eadb5a7d368dfc05d7781b"
server_key_iv: "a28e169ee71ea1b01f5ceeba8854c1d6"
server_key_version: "encryptionkey@0"
I20260620 22:58:45.544886 3765 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260620 22:58:45.548309 3780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:45.549650 3765 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260620 22:58:45.549803 3765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "1f15c8a4b0cb417fb4a84764af301e02"
format_stamp: "Formatted at 2026-06-20 22:58:45 on dist-test-slave-f63b"
server_key: "e4a98835d7eadb5a7d368dfc05d7781b"
server_key_iv: "a28e169ee71ea1b01f5ceeba8854c1d6"
server_key_version: "encryptionkey@0"
I20260620 22:58:45.550004 3765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:58:45.591995 3765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:58:45.596195 3765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:58:45.596514 3765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:58:45.597548 3765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:58:45.599005 3765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:58:45.599077 3765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:45.599201 3765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:58:45.599259 3765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:45.632104 3765 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:36621
I20260620 22:58:45.632150 3892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:36621 every 8 connection(s)
I20260620 22:58:45.633371 3765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 22:58:45.635268 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 3765
I20260620 22:58:45.635468 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance
I20260620 22:58:45.635915 2996 external_mini_cluster.cc:1428] Setting key ce83a21ffdc0f170571ca7d62ffd5231
Jun 20 22:58:45 dist-test-slave-f63b krb5kdc[3525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996325, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.237.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.3@KRBTEST.COM" created.
I20260620 22:58:45.657541 3761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:45.636211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:50173 (local address 127.2.237.62:38775)
0620 22:58:45.636424 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:45.636430 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:45.637835 (+ 1405us) server_negotiation.cc:407] Connection header received
0620 22:58:45.640716 (+ 2881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:45.640720 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:45.640783 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:45.640868 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:45.643059 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:45.644144 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:45.645670 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:45.646463 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:45.646631 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:45.650435 (+ 3804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:45.650444 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:45.650451 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:45.650500 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:45.654096 (+ 3596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:45.654968 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:45.654972 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:45.654975 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:45.655103 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:45.655632 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:45.655635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:45.655637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:45.655880 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:45.655996 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:45.657026 (+ 1030us) server_negotiation.cc:299] Negotiation successful
0620 22:58:45.657295 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260620 22:58:45.659128 3895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:45.636789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:38775 (local address 127.2.237.2:50173)
0620 22:58:45.637663 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:45.637712 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:45.640415 (+ 2703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:45.641177 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:45.641193 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:45.641996 (+ 803us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:58:45.642873 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:45.642902 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:45.644291 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:45.644295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:45.645527 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:45.645542 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:45.646733 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:45.646739 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:45.646939 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:45.647639 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:58:45.647661 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:58:45.650180 (+ 2519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:58:45.654221 (+ 4041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:45.654228 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:45.654240 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:45.654863 (+ 623us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:45.655226 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:45.655239 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:45.655241 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:45.655518 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:45.656144 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:58:45.656151 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:58:45.656563 (+ 412us) client_negotiation.cc:769] Sending connection context
0620 22:58:45.656882 (+ 319us) client_negotiation.cc:240] Negotiation successful
0620 22:58:45.657377 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":438,"threads_started":1}
I20260620 22:58:45.660784 3893 heartbeater.cc:347] Connected to a master server at 127.2.237.62:38775
I20260620 22:58:45.661139 3893 heartbeater.cc:464] Registering TS with master...
I20260620 22:58:45.662076 3893 heartbeater.cc:511] Master 127.2.237.62:38775 requested a full tablet report, sending...
I20260620 22:58:45.663838 3570 ts_manager.cc:194] Registered new tserver with Master: 1f15c8a4b0cb417fb4a84764af301e02 (127.2.237.2:36621)
I20260620 22:58:45.665112 3570 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.2@KRBTEST.COM'} at 127.2.237.2:50173
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
I20260620 22:58:45.681344 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:0
--local_ip_for_outbound_sockets=127.2.237.3
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:38775
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:43795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:58:46.001628 3898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:58:46.001920 3898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:58:46.002077 3898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:58:46.009305 3898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:58:46.009444 3898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:58:46.009622 3898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 22:58:46.019537 3898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:43795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:38775
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:58:46.021036 3898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:58:46.022387 3898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:58:46.034965 3904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:46.035077 3903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:46.035185 3906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:58:46.036087 3898 server_base.cc:1034] running on GCE node
I20260620 22:58:46.036725 3898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:58:46.037645 3898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:58:46.038884 3898 hybrid_clock.cc:648] HybridClock initialized: now 1781996326038845 us; error 69 us; skew 500 ppm
I20260620 22:58:46.039151 3898 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:58:46 dist-test-slave-f63b krb5kdc[3525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996326, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:58:46.042219 3898 init.cc:377] Logged in from keytab as kudu/127.2.237.3@KRBTEST.COM (short username kudu)
I20260620 22:58:46.043596 3898 webserver.cc:460] Webserver started at http://127.2.237.3:43891/ using document root <none> and password file <none>
I20260620 22:58:46.044284 3898 fs_manager.cc:359] Metadata directory not provided
I20260620 22:58:46.044430 3898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:58:46.044762 3898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:58:46.047891 3898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-2/data/instance:
uuid: "0425d5f519dc4da2babdc3eac32a1e89"
format_stamp: "Formatted at 2026-06-20 22:58:46 on dist-test-slave-f63b"
server_key: "d346078170f4ee03a9b75c96f6c1cdd6"
server_key_iv: "0317bbab46fe6efe4a2a8fb67bed84dc"
server_key_version: "encryptionkey@0"
I20260620 22:58:46.048691 3898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance:
uuid: "0425d5f519dc4da2babdc3eac32a1e89"
format_stamp: "Formatted at 2026-06-20 22:58:46 on dist-test-slave-f63b"
server_key: "d346078170f4ee03a9b75c96f6c1cdd6"
server_key_iv: "0317bbab46fe6efe4a2a8fb67bed84dc"
server_key_version: "encryptionkey@0"
I20260620 22:58:46.054010 3898 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260620 22:58:46.058807 3913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:46.060527 3898 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260620 22:58:46.060704 3898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "0425d5f519dc4da2babdc3eac32a1e89"
format_stamp: "Formatted at 2026-06-20 22:58:46 on dist-test-slave-f63b"
server_key: "d346078170f4ee03a9b75c96f6c1cdd6"
server_key_iv: "0317bbab46fe6efe4a2a8fb67bed84dc"
server_key_version: "encryptionkey@0"
I20260620 22:58:46.060956 3898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:58:46.073848 3898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:58:46.078460 3898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:58:46.078790 3898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:58:46.079775 3898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:58:46.081229 3898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:58:46.081327 3898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:46.081477 3898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:58:46.081549 3898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:46.114414 3898 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:37703
I20260620 22:58:46.114457 4025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:37703 every 8 connection(s)
I20260620 22:58:46.115741 3898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 22:58:46.117676 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 3898
I20260620 22:58:46.117887 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance
I20260620 22:58:46.118458 2996 external_mini_cluster.cc:1428] Setting key f96c2dab5adec429839d76bcdcebe7fc
Jun 20 22:58:46 dist-test-slave-f63b krb5kdc[3525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996326, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:58:46.136991 3761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:46.118620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:36027 (local address 127.2.237.62:38775)
0620 22:58:46.118799 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:46.118803 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:46.120384 (+ 1581us) server_negotiation.cc:407] Connection header received
0620 22:58:46.122393 (+ 2009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:46.122398 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:46.122463 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:46.122545 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:46.124176 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.125293 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.126882 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.127559 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.127681 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:46.131434 (+ 3753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:46.131438 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:46.131441 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:46.131473 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:46.134351 (+ 2878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:46.135054 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:46.135058 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:46.135060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:46.135175 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:46.135569 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:46.135572 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:46.135574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:46.135817 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:46.135934 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:46.136573 (+ 639us) server_negotiation.cc:299] Negotiation successful
0620 22:58:46.136723 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260620 22:58:46.137861 4028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:46.119176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:38775 (local address 127.2.237.3:36027)
0620 22:58:46.120221 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:46.120272 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:46.122143 (+ 1871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:46.122819 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:46.122827 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:46.123341 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:58:46.123986 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:46.124004 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.125441 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.125445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:46.126742 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:46.126758 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.127751 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.127756 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:46.128019 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:46.128725 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:58:46.128746 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:58:46.131234 (+ 2488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:58:46.134465 (+ 3231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:46.134471 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:46.134494 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:46.134948 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:46.135278 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:46.135283 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:46.135286 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:46.135461 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:46.135949 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:58:46.135955 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:58:46.136253 (+ 298us) client_negotiation.cc:769] Sending connection context
0620 22:58:46.136463 (+ 210us) client_negotiation.cc:240] Negotiation successful
0620 22:58:46.136839 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":755,"thread_start_us":359,"threads_started":1}
I20260620 22:58:46.139221 4026 heartbeater.cc:347] Connected to a master server at 127.2.237.62:38775
I20260620 22:58:46.139500 4026 heartbeater.cc:464] Registering TS with master...
I20260620 22:58:46.140098 4026 heartbeater.cc:511] Master 127.2.237.62:38775 requested a full tablet report, sending...
I20260620 22:58:46.141397 3570 ts_manager.cc:194] Registered new tserver with Master: 0425d5f519dc4da2babdc3eac32a1e89 (127.2.237.3:37703)
I20260620 22:58:46.142550 3570 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.3@KRBTEST.COM'} at 127.2.237.3:36027
I20260620 22:58:46.146296 2996 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 22:58:46.160748 3759 heartbeater.cc:503] Master 127.2.237.62:38775 was elected leader, sending a full tablet report...
I20260620 22:58:46.166214 3761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:46.152332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36684 (local address 127.2.237.62:38775)
0620 22:58:46.152497 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:46.152501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:46.153523 (+ 1022us) server_negotiation.cc:407] Connection header received
0620 22:58:46.153676 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:46.153680 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:46.153741 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:46.153834 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:46.155255 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.155961 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.157181 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.157606 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.157703 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:46.159966 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:46.159970 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:46.159973 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:46.160002 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:46.163307 (+ 3305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:46.164049 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:46.164057 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:46.164060 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:46.164219 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:46.164707 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:46.164712 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:46.164716 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:46.165261 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:46.165454 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:46.165731 (+ 277us) server_negotiation.cc:299] Negotiation successful
0620 22:58:46.165892 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260620 22:58:46.172376 3570 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36684:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 22:58:46.179742 3570 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 22:58:46.217352 4039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:46.191489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:36621 (local address 127.0.0.1:33200)
0620 22:58:46.203747 (+ 12258us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:46.203814 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:46.204043 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:46.209433 (+ 5390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:46.209443 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:46.209483 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:58:46.210383 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:46.210396 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.212647 (+ 2251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.212655 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:46.214688 (+ 2033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:46.214700 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.216408 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.216413 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:46.216589 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:46.216643 (+ 54us) client_negotiation.cc:769] Sending connection context
0620 22:58:46.216798 (+ 155us) client_negotiation.cc:240] Negotiation successful
0620 22:58:46.217007 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11921,"thread_start_us":16822,"threads_started":1}
I20260620 22:58:46.217352 4040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:46.191680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33200 (local address 127.2.237.2:36621)
0620 22:58:46.208389 (+ 16709us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:46.208399 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:46.208428 (+ 29us) server_negotiation.cc:407] Connection header received
0620 22:58:46.208548 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:46.208556 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:46.208845 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:46.209065 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:58:46.210607 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.212415 (+ 1808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.214834 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.216219 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.216382 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:46.216600 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:46.216838 (+ 238us) server_negotiation.cc:299] Negotiation successful
0620 22:58:46.217007 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16556,"thread_start_us":16429,"threads_started":1}
I20260620 22:58:46.218405 4041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:46.192040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:37541 (local address 127.0.0.1:34170)
0620 22:58:46.210251 (+ 18211us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:46.210273 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:46.210405 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:46.211167 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:46.211171 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:46.211195 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:58:46.211670 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:46.211683 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.213264 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.213270 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:46.215487 (+ 2217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:46.215501 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.217634 (+ 2133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.217639 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:46.217792 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:46.217821 (+ 29us) client_negotiation.cc:769] Sending connection context
0620 22:58:46.217922 (+ 101us) client_negotiation.cc:240] Negotiation successful
0620 22:58:46.218062 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18073,"thread_start_us":17518,"threads_started":1}
I20260620 22:58:46.218472 4043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:46.192633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34170 (local address 127.2.237.1:37541)
0620 22:58:46.208282 (+ 15649us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:46.208297 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:46.210556 (+ 2259us) server_negotiation.cc:407] Connection header received
0620 22:58:46.210697 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:46.210705 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:46.210954 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:46.211158 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:58:46.211910 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.213128 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.215678 (+ 2550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.217137 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.217260 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:46.217485 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:46.218007 (+ 522us) server_negotiation.cc:299] Negotiation successful
0620 22:58:46.218171 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15459,"thread_start_us":15298,"threads_started":1}
I20260620 22:58:46.219214 4038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:46.191635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:37703 (local address 127.0.0.1:53228)
0620 22:58:46.207693 (+ 16058us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:46.207719 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:46.207852 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:46.211412 (+ 3560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:46.211416 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:46.211445 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:58:46.211911 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:46.211923 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.213797 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.213820 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:46.215989 (+ 2169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:46.216000 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.218020 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.218026 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:46.218191 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:46.218222 (+ 31us) client_negotiation.cc:769] Sending connection context
0620 22:58:46.218306 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 22:58:46.218444 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15886,"thread_start_us":11612,"threads_started":1}
I20260620 22:58:46.220112 4042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:46.192219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53228 (local address 127.2.237.3:37703)
0620 22:58:46.210700 (+ 18481us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:46.210706 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:46.210733 (+ 27us) server_negotiation.cc:407] Connection header received
0620 22:58:46.210811 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:46.210818 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:46.210989 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:46.211205 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:58:46.212104 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.213649 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.216169 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.217547 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.217669 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:46.217893 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:46.219549 (+ 1656us) server_negotiation.cc:299] Negotiation successful
0620 22:58:46.219713 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18314,"thread_start_us":18169,"threads_started":1}
I20260620 22:58:46.222088 3828 tablet_service.cc:1467] Processing CreateTablet for tablet 2928e0fdd66f411dac137ef5c7e1d53d (DEFAULT_TABLE table=test-table [id=10f30f103fd2416da898d1656e6885df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:58:46.222488 3694 tablet_service.cc:1467] Processing CreateTablet for tablet 2928e0fdd66f411dac137ef5c7e1d53d (DEFAULT_TABLE table=test-table [id=10f30f103fd2416da898d1656e6885df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:58:46.222951 3961 tablet_service.cc:1467] Processing CreateTablet for tablet 2928e0fdd66f411dac137ef5c7e1d53d (DEFAULT_TABLE table=test-table [id=10f30f103fd2416da898d1656e6885df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:58:46.223634 3828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2928e0fdd66f411dac137ef5c7e1d53d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:58:46.223855 3694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2928e0fdd66f411dac137ef5c7e1d53d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:58:46.224184 3961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2928e0fdd66f411dac137ef5c7e1d53d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:58:46.239967 4044 tablet_bootstrap.cc:492] T 2928e0fdd66f411dac137ef5c7e1d53d P 0425d5f519dc4da2babdc3eac32a1e89: Bootstrap starting.
I20260620 22:58:46.242610 4045 tablet_bootstrap.cc:492] T 2928e0fdd66f411dac137ef5c7e1d53d P b0187ee975cc4a5d8bf65fd98fb29ad6: Bootstrap starting.
I20260620 22:58:46.244046 4046 tablet_bootstrap.cc:492] T 2928e0fdd66f411dac137ef5c7e1d53d P 1f15c8a4b0cb417fb4a84764af301e02: Bootstrap starting.
I20260620 22:58:46.244233 4044 tablet_bootstrap.cc:654] T 2928e0fdd66f411dac137ef5c7e1d53d P 0425d5f519dc4da2babdc3eac32a1e89: Neither blocks nor log segments found. Creating new log.
I20260620 22:58:46.245587 4044 log.cc:824] T 2928e0fdd66f411dac137ef5c7e1d53d P 0425d5f519dc4da2babdc3eac32a1e89: Log is configured to *not* fsync() on all Append() calls
I20260620 22:58:46.246188 4045 tablet_bootstrap.cc:654] T 2928e0fdd66f411dac137ef5c7e1d53d P b0187ee975cc4a5d8bf65fd98fb29ad6: Neither blocks nor log segments found. Creating new log.
I20260620 22:58:46.247937 4046 tablet_bootstrap.cc:654] T 2928e0fdd66f411dac137ef5c7e1d53d P 1f15c8a4b0cb417fb4a84764af301e02: Neither blocks nor log segments found. Creating new log.
I20260620 22:58:46.248231 4045 log.cc:824] T 2928e0fdd66f411dac137ef5c7e1d53d P b0187ee975cc4a5d8bf65fd98fb29ad6: Log is configured to *not* fsync() on all Append() calls
I20260620 22:58:46.249228 4046 log.cc:824] T 2928e0fdd66f411dac137ef5c7e1d53d P 1f15c8a4b0cb417fb4a84764af301e02: Log is configured to *not* fsync() on all Append() calls
I20260620 22:58:46.249847 4044 tablet_bootstrap.cc:492] T 2928e0fdd66f411dac137ef5c7e1d53d P 0425d5f519dc4da2babdc3eac32a1e89: No bootstrap required, opened a new log
I20260620 22:58:46.250128 4044 ts_tablet_manager.cc:1397] T 2928e0fdd66f411dac137ef5c7e1d53d P 0425d5f519dc4da2babdc3eac32a1e89: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260620 22:58:46.251525 4045 tablet_bootstrap.cc:492] T 2928e0fdd66f411dac137ef5c7e1d53d P b0187ee975cc4a5d8bf65fd98fb29ad6: No bootstrap required, opened a new log
I20260620 22:58:46.251798 4045 ts_tablet_manager.cc:1397] T 2928e0fdd66f411dac137ef5c7e1d53d P b0187ee975cc4a5d8bf65fd98fb29ad6: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260620 22:58:46.252790 4046 tablet_bootstrap.cc:492] T 2928e0fdd66f411dac137ef5c7e1d53d P 1f15c8a4b0cb417fb4a84764af301e02: No bootstrap required, opened a new log
I20260620 22:58:46.253044 4046 ts_tablet_manager.cc:1397] T 2928e0fdd66f411dac137ef5c7e1d53d P 1f15c8a4b0cb417fb4a84764af301e02: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260620 22:58:46.261469 4044 raft_consensus.cc:348] T 2928e0fdd66f411dac137ef5c7e1d53d P 0425d5f519dc4da2babdc3eac32a1e89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f15c8a4b0cb417fb4a84764af301e02" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 36621 } } peers { permanent_uuid: "0425d5f519dc4da2babdc3eac32a1e89" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 37703 } } peers { permanent_uuid: "b0187ee975cc4a5d8bf65fd98fb29ad6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 37541 } }
I20260620 22:58:46.261941 4044 raft_consensus.cc:374] T 2928e0fdd66f411dac137ef5c7e1d53d P 0425d5f519dc4da2babdc3eac32a1e89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:58:46.262036 4044 raft_consensus.cc:729] T 2928e0fdd66f411dac137ef5c7e1d53d P 0425d5f519dc4da2babdc3eac32a1e89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0425d5f519dc4da2babdc3eac32a1e89, State: Initialized, Role: FOLLOWER
I20260620 22:58:46.262040 4046 raft_consensus.cc:348] T 2928e0fdd66f411dac137ef5c7e1d53d P 1f15c8a4b0cb417fb4a84764af301e02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f15c8a4b0cb417fb4a84764af301e02" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 36621 } } peers { permanent_uuid: "0425d5f519dc4da2babdc3eac32a1e89" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 37703 } } peers { permanent_uuid: "b0187ee975cc4a5d8bf65fd98fb29ad6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 37541 } }
I20260620 22:58:46.262822 4046 raft_consensus.cc:374] T 2928e0fdd66f411dac137ef5c7e1d53d P 1f15c8a4b0cb417fb4a84764af301e02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:58:46.262756 4044 consensus_queue.cc:260] T 2928e0fdd66f411dac137ef5c7e1d53d P 0425d5f519dc4da2babdc3eac32a1e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f15c8a4b0cb417fb4a84764af301e02" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 36621 } } peers { permanent_uuid: "0425d5f519dc4da2babdc3eac32a1e89" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 37703 } } peers { permanent_uuid: "b0187ee975cc4a5d8bf65fd98fb29ad6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 37541 } }
I20260620 22:58:46.262641 4045 raft_consensus.cc:348] T 2928e0fdd66f411dac137ef5c7e1d53d P b0187ee975cc4a5d8bf65fd98fb29ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f15c8a4b0cb417fb4a84764af301e02" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 36621 } } peers { permanent_uuid: "0425d5f519dc4da2babdc3eac32a1e89" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 37703 } } peers { permanent_uuid: "b0187ee975cc4a5d8bf65fd98fb29ad6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 37541 } }
I20260620 22:58:46.263053 4045 raft_consensus.cc:374] T 2928e0fdd66f411dac137ef5c7e1d53d P b0187ee975cc4a5d8bf65fd98fb29ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:58:46.263147 4045 raft_consensus.cc:729] T 2928e0fdd66f411dac137ef5c7e1d53d P b0187ee975cc4a5d8bf65fd98fb29ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0187ee975cc4a5d8bf65fd98fb29ad6, State: Initialized, Role: FOLLOWER
I20260620 22:58:46.262923 4046 raft_consensus.cc:729] T 2928e0fdd66f411dac137ef5c7e1d53d P 1f15c8a4b0cb417fb4a84764af301e02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f15c8a4b0cb417fb4a84764af301e02, State: Initialized, Role: FOLLOWER
I20260620 22:58:46.264101 4046 consensus_queue.cc:260] T 2928e0fdd66f411dac137ef5c7e1d53d P 1f15c8a4b0cb417fb4a84764af301e02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f15c8a4b0cb417fb4a84764af301e02" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 36621 } } peers { permanent_uuid: "0425d5f519dc4da2babdc3eac32a1e89" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 37703 } } peers { permanent_uuid: "b0187ee975cc4a5d8bf65fd98fb29ad6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 37541 } }
I20260620 22:58:46.263983 4045 consensus_queue.cc:260] T 2928e0fdd66f411dac137ef5c7e1d53d P b0187ee975cc4a5d8bf65fd98fb29ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f15c8a4b0cb417fb4a84764af301e02" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 36621 } } peers { permanent_uuid: "0425d5f519dc4da2babdc3eac32a1e89" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 37703 } } peers { permanent_uuid: "b0187ee975cc4a5d8bf65fd98fb29ad6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 37541 } }
I20260620 22:58:46.265353 3893 heartbeater.cc:503] Master 127.2.237.62:38775 was elected leader, sending a full tablet report...
I20260620 22:58:46.266001 4046 ts_tablet_manager.cc:1428] T 2928e0fdd66f411dac137ef5c7e1d53d P 1f15c8a4b0cb417fb4a84764af301e02: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260620 22:58:46.268167 4045 ts_tablet_manager.cc:1428] T 2928e0fdd66f411dac137ef5c7e1d53d P b0187ee975cc4a5d8bf65fd98fb29ad6: Time spent starting tablet: real 0.016s user 0.008s sys 0.006s
I20260620 22:58:46.268994 4026 heartbeater.cc:503] Master 127.2.237.62:38775 was elected leader, sending a full tablet report...
I20260620 22:58:46.269658 4044 ts_tablet_manager.cc:1428] T 2928e0fdd66f411dac137ef5c7e1d53d P 0425d5f519dc4da2babdc3eac32a1e89: Time spent starting tablet: real 0.019s user 0.018s sys 0.001s
I20260620 22:58:46.339176 4052 raft_consensus.cc:482] T 2928e0fdd66f411dac137ef5c7e1d53d P b0187ee975cc4a5d8bf65fd98fb29ad6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 22:58:46.340065 4052 raft_consensus.cc:504] T 2928e0fdd66f411dac137ef5c7e1d53d P b0187ee975cc4a5d8bf65fd98fb29ad6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f15c8a4b0cb417fb4a84764af301e02" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 36621 } } peers { permanent_uuid: "0425d5f519dc4da2babdc3eac32a1e89" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 37703 } } peers { permanent_uuid: "b0187ee975cc4a5d8bf65fd98fb29ad6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 37541 } }
I20260620 22:58:46.341956 4052 leader_election.cc:288] T 2928e0fdd66f411dac137ef5c7e1d53d P b0187ee975cc4a5d8bf65fd98fb29ad6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f15c8a4b0cb417fb4a84764af301e02 (127.2.237.2:36621), 0425d5f519dc4da2babdc3eac32a1e89 (127.2.237.3:37703)
I20260620 22:58:46.348798 4042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:46.342624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:54841 (local address 127.2.237.3:37703)
0620 22:58:46.342889 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:46.342899 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:46.343486 (+ 587us) server_negotiation.cc:407] Connection header received
0620 22:58:46.343692 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:46.343701 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:46.343760 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:46.343892 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:58:46.344469 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.345228 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.347046 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.347850 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.347997 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:46.348111 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:46.348391 (+ 280us) server_negotiation.cc:299] Negotiation successful
0620 22:58:46.348531 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260620 22:58:46.349146 4053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:46.342388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:37703 (local address 127.2.237.1:54841)
0620 22:58:46.343358 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:46.343380 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:46.343561 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:46.343907 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:46.343911 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:46.343943 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:58:46.344321 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:46.344330 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.345376 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.345384 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:46.346892 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:46.346904 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.348073 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.348079 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:46.348231 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:46.348254 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 22:58:46.348366 (+ 112us) client_negotiation.cc:240] Negotiation successful
0620 22:58:46.348573 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":842,"thread_start_us":645,"threads_started":1}
I20260620 22:58:46.349709 4054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:46.342434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:36621 (local address 127.2.237.1:56103)
0620 22:58:46.343468 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:46.343489 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:46.343609 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:46.343943 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:46.343947 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:46.343968 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:58:46.344369 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:46.344378 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.345825 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.345831 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:46.347243 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:46.347254 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.348635 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.348644 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:46.348767 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:46.348790 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 22:58:46.348894 (+ 104us) client_negotiation.cc:240] Negotiation successful
0620 22:58:46.349141 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":893,"mutex_wait_us":3,"thread_start_us":694,"threads_started":1}
I20260620 22:58:46.350283 4040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:46.342514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:56103 (local address 127.2.237.2:36621)
0620 22:58:46.342783 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:46.342793 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:46.343564 (+ 771us) server_negotiation.cc:407] Connection header received
0620 22:58:46.343769 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:46.343773 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:46.343828 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:46.343931 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:58:46.344545 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.345686 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.347412 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.348349 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.348509 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:46.348613 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:46.348918 (+ 305us) server_negotiation.cc:299] Negotiation successful
0620 22:58:46.349050 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260620 22:58:46.350553 3981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2928e0fdd66f411dac137ef5c7e1d53d" candidate_uuid: "b0187ee975cc4a5d8bf65fd98fb29ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0425d5f519dc4da2babdc3eac32a1e89" is_pre_election: true
I20260620 22:58:46.351146 3981 raft_consensus.cc:2454] T 2928e0fdd66f411dac137ef5c7e1d53d P 0425d5f519dc4da2babdc3eac32a1e89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0187ee975cc4a5d8bf65fd98fb29ad6 in term 0.
I20260620 22:58:46.351200 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2928e0fdd66f411dac137ef5c7e1d53d" candidate_uuid: "b0187ee975cc4a5d8bf65fd98fb29ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f15c8a4b0cb417fb4a84764af301e02" is_pre_election: true
I20260620 22:58:46.351662 3848 raft_consensus.cc:2454] T 2928e0fdd66f411dac137ef5c7e1d53d P 1f15c8a4b0cb417fb4a84764af301e02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0187ee975cc4a5d8bf65fd98fb29ad6 in term 0.
I20260620 22:58:46.351884 3648 leader_election.cc:302] T 2928e0fdd66f411dac137ef5c7e1d53d P b0187ee975cc4a5d8bf65fd98fb29ad6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0425d5f519dc4da2babdc3eac32a1e89, b0187ee975cc4a5d8bf65fd98fb29ad6; no voters:
I20260620 22:58:46.352291 4052 raft_consensus.cc:2781] T 2928e0fdd66f411dac137ef5c7e1d53d P b0187ee975cc4a5d8bf65fd98fb29ad6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 22:58:46.352442 4052 raft_consensus.cc:482] T 2928e0fdd66f411dac137ef5c7e1d53d P b0187ee975cc4a5d8bf65fd98fb29ad6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 22:58:46.352557 4052 raft_consensus.cc:3037] T 2928e0fdd66f411dac137ef5c7e1d53d P b0187ee975cc4a5d8bf65fd98fb29ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:58:46.355154 4052 raft_consensus.cc:504] T 2928e0fdd66f411dac137ef5c7e1d53d P b0187ee975cc4a5d8bf65fd98fb29ad6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f15c8a4b0cb417fb4a84764af301e02" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 36621 } } peers { permanent_uuid: "0425d5f519dc4da2babdc3eac32a1e89" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 37703 } } peers { permanent_uuid: "b0187ee975cc4a5d8bf65fd98fb29ad6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 37541 } }
I20260620 22:58:46.355932 4052 leader_election.cc:288] T 2928e0fdd66f411dac137ef5c7e1d53d P b0187ee975cc4a5d8bf65fd98fb29ad6 [CANDIDATE]: Term 1 election: Requested vote from peers 1f15c8a4b0cb417fb4a84764af301e02 (127.2.237.2:36621), 0425d5f519dc4da2babdc3eac32a1e89 (127.2.237.3:37703)
I20260620 22:58:46.356412 3981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2928e0fdd66f411dac137ef5c7e1d53d" candidate_uuid: "b0187ee975cc4a5d8bf65fd98fb29ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0425d5f519dc4da2babdc3eac32a1e89"
I20260620 22:58:46.356397 3848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2928e0fdd66f411dac137ef5c7e1d53d" candidate_uuid: "b0187ee975cc4a5d8bf65fd98fb29ad6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f15c8a4b0cb417fb4a84764af301e02"
I20260620 22:58:46.356611 3981 raft_consensus.cc:3037] T 2928e0fdd66f411dac137ef5c7e1d53d P 0425d5f519dc4da2babdc3eac32a1e89 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:58:46.356611 3848 raft_consensus.cc:3037] T 2928e0fdd66f411dac137ef5c7e1d53d P 1f15c8a4b0cb417fb4a84764af301e02 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:58:46.358803 3981 raft_consensus.cc:2454] T 2928e0fdd66f411dac137ef5c7e1d53d P 0425d5f519dc4da2babdc3eac32a1e89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0187ee975cc4a5d8bf65fd98fb29ad6 in term 1.
I20260620 22:58:46.359225 3648 leader_election.cc:302] T 2928e0fdd66f411dac137ef5c7e1d53d P b0187ee975cc4a5d8bf65fd98fb29ad6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0425d5f519dc4da2babdc3eac32a1e89, b0187ee975cc4a5d8bf65fd98fb29ad6; no voters:
I20260620 22:58:46.359512 4052 raft_consensus.cc:2781] T 2928e0fdd66f411dac137ef5c7e1d53d P b0187ee975cc4a5d8bf65fd98fb29ad6 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:58:46.360037 4052 raft_consensus.cc:686] T 2928e0fdd66f411dac137ef5c7e1d53d P b0187ee975cc4a5d8bf65fd98fb29ad6 [term 1 LEADER]: Becoming Leader. State: Replica: b0187ee975cc4a5d8bf65fd98fb29ad6, State: Running, Role: LEADER
I20260620 22:58:46.360532 4052 consensus_queue.cc:237] T 2928e0fdd66f411dac137ef5c7e1d53d P b0187ee975cc4a5d8bf65fd98fb29ad6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f15c8a4b0cb417fb4a84764af301e02" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 36621 } } peers { permanent_uuid: "0425d5f519dc4da2babdc3eac32a1e89" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 37703 } } peers { permanent_uuid: "b0187ee975cc4a5d8bf65fd98fb29ad6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 37541 } }
I20260620 22:58:46.365520 3570 catalog_manager.cc:5495] T 2928e0fdd66f411dac137ef5c7e1d53d P b0187ee975cc4a5d8bf65fd98fb29ad6 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0187ee975cc4a5d8bf65fd98fb29ad6 (127.2.237.1). New cstate: current_term: 1 leader_uuid: "b0187ee975cc4a5d8bf65fd98fb29ad6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f15c8a4b0cb417fb4a84764af301e02" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 36621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0425d5f519dc4da2babdc3eac32a1e89" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 37703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0187ee975cc4a5d8bf65fd98fb29ad6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 37541 } health_report { overall_health: HEALTHY } } }
W20260620 22:58:46.367450 4027 tablet.cc:2369] T 2928e0fdd66f411dac137ef5c7e1d53d P 0425d5f519dc4da2babdc3eac32a1e89: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 22:58:46.378229 3760 tablet.cc:2369] T 2928e0fdd66f411dac137ef5c7e1d53d P b0187ee975cc4a5d8bf65fd98fb29ad6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 22:58:46.385628 3894 tablet.cc:2369] T 2928e0fdd66f411dac137ef5c7e1d53d P 1f15c8a4b0cb417fb4a84764af301e02: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 20 22:58:46 dist-test-slave-f63b krb5kdc[3525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.1@KRBTEST.COM
I20260620 22:58:46.417115 4043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:46.401681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34176 (local address 127.2.237.1:37541)
0620 22:58:46.401877 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:46.401881 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:46.402402 (+ 521us) server_negotiation.cc:407] Connection header received
0620 22:58:46.402639 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:46.402646 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:46.402719 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:46.402828 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:46.404713 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.405492 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.406766 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.407331 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.407466 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:46.411927 (+ 4461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:46.411941 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:46.411957 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:46.412008 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:46.414771 (+ 2763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:46.415362 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:46.415369 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:46.415371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:46.415505 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:46.415803 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:46.415806 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:46.415808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:46.416185 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:46.416394 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:46.416653 (+ 259us) server_negotiation.cc:299] Negotiation successful
0620 22:58:46.416835 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Jun 20 22:58:46 dist-test-slave-f63b krb5kdc[3525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996326, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
I20260620 22:58:46.457580 3848 raft_consensus.cc:2454] T 2928e0fdd66f411dac137ef5c7e1d53d P 1f15c8a4b0cb417fb4a84764af301e02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0187ee975cc4a5d8bf65fd98fb29ad6 in term 1.
Jun 20 22:58:46 dist-test-slave-f63b krb5kdc[3525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996326, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.237.1@KRBTEST.COM
I20260620 22:58:46.475101 4043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:46.459868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40760 (local address 127.2.237.1:37541)
0620 22:58:46.460128 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:46.460136 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:46.460417 (+ 281us) server_negotiation.cc:407] Connection header received
0620 22:58:46.460635 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:46.460640 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:46.460703 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:46.460809 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:46.462335 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.463239 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.464590 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.464982 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.465133 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:46.469620 (+ 4487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:46.469628 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:46.469635 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:46.469686 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:46.472617 (+ 2931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:46.473261 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:46.473265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:46.473268 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:46.473406 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:46.473841 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:46.473844 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:46.473847 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:46.474105 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:46.474326 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:46.474507 (+ 181us) server_negotiation.cc:299] Negotiation successful
0620 22:58:46.474760 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
Jun 20 22:58:46 dist-test-slave-f63b krb5kdc[3525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 22:58:46 dist-test-slave-f63b krb5kdc[3525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.1@KRBTEST.COM
I20260620 22:58:46.579716 4043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:46.563665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40770 (local address 127.2.237.1:37541)
0620 22:58:46.563924 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:46.563931 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:46.564321 (+ 390us) server_negotiation.cc:407] Connection header received
0620 22:58:46.564531 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:46.564535 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:46.564616 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:46.564703 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:46.566929 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.567743 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.568939 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.569768 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.569871 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:46.573771 (+ 3900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:46.573776 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:46.573779 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:46.573814 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:46.576622 (+ 2808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:46.577245 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:46.577250 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:46.577252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:46.577375 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:46.577675 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:46.577679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:46.577681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:46.577983 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:46.578164 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:46.579145 (+ 981us) server_negotiation.cc:299] Negotiation successful
0620 22:58:46.579334 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260620 22:58:46.581672 3674 tablet_service.cc:2908] Rejecting scan request for tablet 2928e0fdd66f411dac137ef5c7e1d53d: Uninitialized: clean time has not yet been initialized
I20260620 22:58:46.584631 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 3631
I20260620 22:58:46.597215 4043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:46.585500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40776 (local address 127.2.237.1:37541)
0620 22:58:46.585696 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:46.585704 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:46.586104 (+ 400us) server_negotiation.cc:407] Connection header received
0620 22:58:46.586226 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:46.586229 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:46.586296 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:46.586379 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:46.587532 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.588278 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.589414 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.589784 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.589880 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:46.592057 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:46.592061 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:46.592064 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:46.592091 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:46.594947 (+ 2856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:46.595579 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:46.595583 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:46.595585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:46.595697 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:46.596000 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:46.596004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:46.596006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:46.596248 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:46.596437 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:46.596703 (+ 266us) server_negotiation.cc:299] Negotiation successful
0620 22:58:46.596894 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260620 22:58:46.597918 3754 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:58:46.693693 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 3631
I20260620 22:58:46.713289 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 3765
Jun 20 22:58:46 dist-test-slave-f63b krb5kdc[3525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.2@KRBTEST.COM
I20260620 22:58:46.728379 4040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:46.713984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57672 (local address 127.2.237.2:36621)
0620 22:58:46.714173 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:46.714180 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:46.714247 (+ 67us) server_negotiation.cc:407] Connection header received
0620 22:58:46.714470 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:46.714482 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:46.714628 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:46.714745 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:46.715932 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.716680 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.717945 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.718343 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.718478 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:46.722621 (+ 4143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:46.722640 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:46.722660 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:46.722702 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:46.725515 (+ 2813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:46.726232 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:46.726248 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:46.726250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:46.726373 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:46.726877 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:46.726881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:46.726883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:46.727258 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:46.727586 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:46.727820 (+ 234us) server_negotiation.cc:299] Negotiation successful
0620 22:58:46.728036 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 22:58:46.729055 3888 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:58:46.819366 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 3765
I20260620 22:58:46.836009 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 3898
Jun 20 22:58:46 dist-test-slave-f63b krb5kdc[3525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.3@KRBTEST.COM
I20260620 22:58:46.850260 4042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:46.836789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39518 (local address 127.2.237.3:37703)
0620 22:58:46.836949 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:46.836954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:46.836971 (+ 17us) server_negotiation.cc:407] Connection header received
0620 22:58:46.837116 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:46.837119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:46.837181 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:46.837268 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:46.838411 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.839191 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.840413 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:46.840775 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:46.840875 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:46.844811 (+ 3936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:46.844825 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:46.844842 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:46.844875 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:46.847617 (+ 2742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:46.848254 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:46.848262 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:46.848265 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:46.848407 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:46.848786 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:46.848800 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:46.848806 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:46.849140 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:46.849419 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:46.849707 (+ 288us) server_negotiation.cc:299] Negotiation successful
0620 22:58:46.849939 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 22:58:46.851102 4021 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:58:46.940078 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 3898
I20260620 22:58:46.956688 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 3538
I20260620 22:58:46.957908 3600 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:58:47.070329 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 3538
2026-06-20T22:58:47Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4252 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 22:58:49 dist-test-slave-f63b krb5kdc[4074](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 22:58:49 dist-test-slave-f63b krb5kdc[4074](info): set up 2 sockets
Jun 20 22:58:49 dist-test-slave-f63b krb5kdc[4074](info): commencing operation
krb5kdc: starting...
W20260620 22:58:50.847913 2996 mini_kdc.cc:121] Time spent starting KDC: real 3.734s user 0.002s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 22:58:51 dist-test-slave-f63b krb5kdc[4074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T22:58:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T22:58:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
I20260620 22:58:51.085724 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:39831
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:36135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:39831
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:58:51.417446 4087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:58:51.417730 4087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:58:51.417855 4087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:58:51.425009 4087 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 22:58:51.425110 4087 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260620 22:58:51.425150 4087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:58:51.425230 4087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:58:51.425302 4087 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 22:58:51.425382 4087 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 22:58:51.435607 4087 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:36135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:39831
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:39831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:58:51.437018 4087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:58:51.438400 4087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:58:51.450439 4092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:51.450542 4095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:51.450518 4093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:58:51.450843 4087 server_base.cc:1034] running on GCE node
I20260620 22:58:51.451939 4087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:58:51.453416 4087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:58:51.454684 4087 hybrid_clock.cc:648] HybridClock initialized: now 1781996331454630 us; error 54 us; skew 500 ppm
I20260620 22:58:51.455013 4087 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:58:51 dist-test-slave-f63b krb5kdc[4074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996331, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:58:51.458279 4087 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 22:58:51.459749 4087 webserver.cc:460] Webserver started at http://127.2.237.62:39749/ using document root <none> and password file <none>
I20260620 22:58:51.460461 4087 fs_manager.cc:359] Metadata directory not provided
I20260620 22:58:51.460590 4087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:58:51.460919 4087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:58:51.463878 4087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "2e2c41911b804b5ea76ef0485b087abd"
format_stamp: "Formatted at 2026-06-20 22:58:51 on dist-test-slave-f63b"
server_key: "201dfd33be381c1cf5fce7e3c3f66ef0"
server_key_iv: "797532bb1d46d9e66e71a5f88c261d19"
server_key_version: "encryptionkey@0"
I20260620 22:58:51.464695 4087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "2e2c41911b804b5ea76ef0485b087abd"
format_stamp: "Formatted at 2026-06-20 22:58:51 on dist-test-slave-f63b"
server_key: "201dfd33be381c1cf5fce7e3c3f66ef0"
server_key_iv: "797532bb1d46d9e66e71a5f88c261d19"
server_key_version: "encryptionkey@0"
I20260620 22:58:51.469758 4087 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260620 22:58:51.473220 4102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:51.474691 4087 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 22:58:51.474864 4087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "2e2c41911b804b5ea76ef0485b087abd"
format_stamp: "Formatted at 2026-06-20 22:58:51 on dist-test-slave-f63b"
server_key: "201dfd33be381c1cf5fce7e3c3f66ef0"
server_key_iv: "797532bb1d46d9e66e71a5f88c261d19"
server_key_version: "encryptionkey@0"
I20260620 22:58:51.475083 4087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:58:51.491528 4087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:58:51.495625 4087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:58:51.495908 4087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:58:51.517602 4087 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:39831
I20260620 22:58:51.517624 4163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:39831 every 8 connection(s)
I20260620 22:58:51.519017 4087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 22:58:51.522897 4164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:58:51.523379 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 4087
I20260620 22:58:51.523592 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 22:58:51.524235 2996 external_mini_cluster.cc:1428] Setting key 0a37d71994123636dfd6cdc9e9dc44da
I20260620 22:58:51.537243 4164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e2c41911b804b5ea76ef0485b087abd: Bootstrap starting.
Jun 20 22:58:51 dist-test-slave-f63b krb5kdc[4074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:58:51.540669 4164 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e2c41911b804b5ea76ef0485b087abd: Neither blocks nor log segments found. Creating new log.
I20260620 22:58:51.542075 4164 log.cc:824] T 00000000000000000000000000000000 P 2e2c41911b804b5ea76ef0485b087abd: Log is configured to *not* fsync() on all Append() calls
I20260620 22:58:51.545315 4164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e2c41911b804b5ea76ef0485b087abd: No bootstrap required, opened a new log
I20260620 22:58:51.548666 4167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:51.526169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43448 (local address 127.2.237.62:39831)
0620 22:58:51.526979 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:51.526994 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:51.527042 (+ 48us) server_negotiation.cc:407] Connection header received
0620 22:58:51.529557 (+ 2515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:51.529587 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:51.530109 (+ 522us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:51.530564 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:51.531864 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:51.533839 (+ 1975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:51.535103 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:51.536108 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:51.536303 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:51.540361 (+ 4058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:51.540373 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:51.540394 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:51.540444 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:51.544217 (+ 3773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:51.544873 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:51.544880 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:51.544888 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:51.545113 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:51.545485 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:51.545489 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:51.545492 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:51.546199 (+ 707us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:51.546500 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:51.546830 (+ 330us) server_negotiation.cc:299] Negotiation successful
0620 22:58:51.547277 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":345,"threads_started":1}
I20260620 22:58:51.554832 4164 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e2c41911b804b5ea76ef0485b087abd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e2c41911b804b5ea76ef0485b087abd" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 39831 } }
I20260620 22:58:51.555155 4164 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e2c41911b804b5ea76ef0485b087abd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:58:51.555226 4164 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e2c41911b804b5ea76ef0485b087abd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e2c41911b804b5ea76ef0485b087abd, State: Initialized, Role: FOLLOWER
I20260620 22:58:51.555732 4164 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e2c41911b804b5ea76ef0485b087abd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e2c41911b804b5ea76ef0485b087abd" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 39831 } }
I20260620 22:58:51.555959 4164 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e2c41911b804b5ea76ef0485b087abd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 22:58:51.556080 4164 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e2c41911b804b5ea76ef0485b087abd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 22:58:51.556198 4164 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e2c41911b804b5ea76ef0485b087abd [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:58:51.558691 4164 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e2c41911b804b5ea76ef0485b087abd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e2c41911b804b5ea76ef0485b087abd" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 39831 } }
I20260620 22:58:51.559134 4164 leader_election.cc:302] T 00000000000000000000000000000000 P 2e2c41911b804b5ea76ef0485b087abd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e2c41911b804b5ea76ef0485b087abd; no voters:
I20260620 22:58:51.559633 4164 leader_election.cc:288] T 00000000000000000000000000000000 P 2e2c41911b804b5ea76ef0485b087abd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 22:58:51.559775 4169 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e2c41911b804b5ea76ef0485b087abd [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:58:51.560436 4169 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e2c41911b804b5ea76ef0485b087abd [term 1 LEADER]: Becoming Leader. State: Replica: 2e2c41911b804b5ea76ef0485b087abd, State: Running, Role: LEADER
I20260620 22:58:51.560983 4169 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e2c41911b804b5ea76ef0485b087abd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e2c41911b804b5ea76ef0485b087abd" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 39831 } }
I20260620 22:58:51.561647 4164 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e2c41911b804b5ea76ef0485b087abd [sys.catalog]: configured and running, proceeding with master startup.
I20260620 22:58:51.563946 4171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e2c41911b804b5ea76ef0485b087abd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e2c41911b804b5ea76ef0485b087abd. Latest consensus state: current_term: 1 leader_uuid: "2e2c41911b804b5ea76ef0485b087abd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e2c41911b804b5ea76ef0485b087abd" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 39831 } } }
I20260620 22:58:51.564375 4171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e2c41911b804b5ea76ef0485b087abd [sys.catalog]: This master's current role is: LEADER
I20260620 22:58:51.565187 4170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e2c41911b804b5ea76ef0485b087abd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e2c41911b804b5ea76ef0485b087abd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e2c41911b804b5ea76ef0485b087abd" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 39831 } } }
I20260620 22:58:51.565418 4170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e2c41911b804b5ea76ef0485b087abd [sys.catalog]: This master's current role is: LEADER
I20260620 22:58:51.566630 4174 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 22:58:51.572233 4174 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 22:58:51.581961 4174 catalog_manager.cc:1347] Generated new cluster ID: e1f4a9dc404c43b99037b7a4f0d0845d
I20260620 22:58:51.582163 4174 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 22:58:51.616729 4174 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 22:58:51.618167 4174 catalog_manager.cc:1504] Loading token signing keys...
I20260620 22:58:51.626910 4174 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e2c41911b804b5ea76ef0485b087abd: Generated new TSK 0
I20260620 22:58:51.627801 4174 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.237.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
I20260620 22:58:51.684538 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:0
--local_ip_for_outbound_sockets=127.2.237.1
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:39831
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:36135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:58:52.010272 4190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:58:52.010629 4190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:58:52.010818 4190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:58:52.018018 4190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:58:52.018188 4190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:58:52.018354 4190 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 22:58:52.018502 4190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 22:58:52.028036 4190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:36135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.237.62:39831
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:58:52.029474 4190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:58:52.030959 4190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:58:52.044147 4196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:52.044118 4198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:52.044430 4195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:58:52.044917 4190 server_base.cc:1034] running on GCE node
I20260620 22:58:52.045555 4190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:58:52.046480 4190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:58:52.047690 4190 hybrid_clock.cc:648] HybridClock initialized: now 1781996332047615 us; error 54 us; skew 500 ppm
I20260620 22:58:52.048028 4190 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:58:52 dist-test-slave-f63b krb5kdc[4074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996332, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:58:52.051152 4190 init.cc:377] Logged in from keytab as kudu/127.2.237.1@KRBTEST.COM (short username kudu)
I20260620 22:58:52.052512 4190 webserver.cc:460] Webserver started at http://127.2.237.1:34545/ using document root <none> and password file <none>
I20260620 22:58:52.053205 4190 fs_manager.cc:359] Metadata directory not provided
I20260620 22:58:52.053329 4190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:58:52.053653 4190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:58:52.056537 4190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-0/data/instance:
uuid: "6d958c8a03d94635a6b46cceff8dbfe8"
format_stamp: "Formatted at 2026-06-20 22:58:52 on dist-test-slave-f63b"
server_key: "c5ea8b3bea71a97332ed9ddfd65644ab"
server_key_iv: "fa4e84048ad4640babd09dbf766dcab1"
server_key_version: "encryptionkey@0"
I20260620 22:58:52.057318 4190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance:
uuid: "6d958c8a03d94635a6b46cceff8dbfe8"
format_stamp: "Formatted at 2026-06-20 22:58:52 on dist-test-slave-f63b"
server_key: "c5ea8b3bea71a97332ed9ddfd65644ab"
server_key_iv: "fa4e84048ad4640babd09dbf766dcab1"
server_key_version: "encryptionkey@0"
I20260620 22:58:52.062522 4190 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260620 22:58:52.066073 4205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:52.067526 4190 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 22:58:52.067687 4190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "6d958c8a03d94635a6b46cceff8dbfe8"
format_stamp: "Formatted at 2026-06-20 22:58:52 on dist-test-slave-f63b"
server_key: "c5ea8b3bea71a97332ed9ddfd65644ab"
server_key_iv: "fa4e84048ad4640babd09dbf766dcab1"
server_key_version: "encryptionkey@0"
I20260620 22:58:52.067988 4190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:58:52.089413 4190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:58:52.093422 4190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:58:52.093715 4190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:58:52.096541 4190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:58:52.096659 4190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:52.096879 4190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:58:52.096966 4190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 22:58:52 dist-test-slave-f63b krb5kdc[4074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996332, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:58:52.121381 4214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:52.097318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:43833 (local address 127.2.237.62:39831)
0620 22:58:52.097934 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:52.097941 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:52.099131 (+ 1190us) server_negotiation.cc:407] Connection header received
0620 22:58:52.101373 (+ 2242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:52.101379 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:52.101471 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:52.101593 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:52.104415 (+ 2822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:52.105398 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:52.108629 (+ 3231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:52.109191 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:52.109354 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:52.114115 (+ 4761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:52.114122 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:52.114127 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:52.114171 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:52.117631 (+ 3460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:52.118654 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:52.118661 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:52.118666 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:52.118816 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:52.119398 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:52.119401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:52.119404 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:52.119743 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:52.119903 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:52.120952 (+ 1049us) server_negotiation.cc:299] Negotiation successful
0620 22:58:52.121122 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":386,"threads_started":1}
I20260620 22:58:52.123649 4215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:52.098053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:39831 (local address 127.2.237.1:43833)
0620 22:58:52.098957 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:52.099001 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:52.101060 (+ 2059us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:52.101941 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:52.101955 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:52.102961 (+ 1006us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:58:52.104179 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:52.104215 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:52.105587 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:52.105594 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:52.108448 (+ 2854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:52.108478 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:52.109472 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:52.109480 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:52.109717 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:52.110597 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:58:52.110647 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:58:52.113857 (+ 3210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:58:52.117785 (+ 3928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:52.117794 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:52.117816 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:52.118451 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:52.118934 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:52.118939 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:52.118943 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:52.119233 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:52.119904 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:58:52.119916 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:58:52.120492 (+ 576us) client_negotiation.cc:769] Sending connection context
0620 22:58:52.120866 (+ 374us) client_negotiation.cc:240] Negotiation successful
0620 22:58:52.121571 (+ 705us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":391,"threads_started":1}
I20260620 22:58:52.139971 4190 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:42623
I20260620 22:58:52.139991 4324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:42623 every 8 connection(s)
I20260620 22:58:52.141297 4190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 22:58:52.143748 4325 heartbeater.cc:347] Connected to a master server at 127.2.237.62:39831
I20260620 22:58:52.143931 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 4190
I20260620 22:58:52.144045 4325 heartbeater.cc:464] Registering TS with master...
I20260620 22:58:52.144168 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance
I20260620 22:58:52.144768 2996 external_mini_cluster.cc:1428] Setting key efc0a111c05b835918c7b7f5fc7c6e81
I20260620 22:58:52.145045 4325 heartbeater.cc:511] Master 127.2.237.62:39831 requested a full tablet report, sending...
I20260620 22:58:52.147933 4119 ts_manager.cc:194] Registered new tserver with Master: 6d958c8a03d94635a6b46cceff8dbfe8 (127.2.237.1:42623)
I20260620 22:58:52.150550 4119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.1@KRBTEST.COM'} at 127.2.237.1:43833
WARNING: no policy specified for kudu/127.2.237.2@KRBTEST.COM; defaulting to no policy
I20260620 22:58:52.163741 4214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:52.141680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:57077 (local address 127.2.237.62:39831)
0620 22:58:52.141909 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:52.141917 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:52.151222 (+ 9305us) server_negotiation.cc:407] Connection header received
0620 22:58:52.151449 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:52.151453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:52.151546 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:52.151651 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:52.152632 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:52.153323 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:52.154690 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:52.155418 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:52.155549 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:52.157180 (+ 1631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:52.157189 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:52.157197 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:52.157239 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:52.160835 (+ 3596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:52.161491 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:52.161500 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:52.161504 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:52.161662 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:52.162295 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:52.162299 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:52.162303 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:52.162715 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:52.162854 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:52.163324 (+ 470us) server_negotiation.cc:299] Negotiation successful
0620 22:58:52.163503 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260620 22:58:52.164057 4326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:52.141656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:39831 (local address 127.2.237.1:57077)
0620 22:58:52.151096 (+ 9440us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:52.151118 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:52.151271 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:52.151694 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:52.151699 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:52.152200 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:58:52.152486 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:52.152495 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:52.153447 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:52.153454 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:52.154508 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:52.154519 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:52.155591 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:52.155595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:52.155705 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:52.156333 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:58:52.156348 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:58:52.156994 (+ 646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:58:52.160977 (+ 3983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:52.160980 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:52.160983 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:52.161370 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:52.161794 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:52.161798 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:52.161800 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:52.162101 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:52.162837 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:58:52.162842 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:58:52.163023 (+ 181us) client_negotiation.cc:769] Sending connection context
0620 22:58:52.163183 (+ 160us) client_negotiation.cc:240] Negotiation successful
0620 22:58:52.163504 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9255,"thread_start_us":9103,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
I20260620 22:58:52.189992 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:0
--local_ip_for_outbound_sockets=127.2.237.2
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:39831
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:36135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:58:52.506881 4330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:58:52.507165 4330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:58:52.507257 4330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:58:52.514334 4330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:58:52.514446 4330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:58:52.514528 4330 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 22:58:52.514626 4330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 22:58:52.524345 4330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:36135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.237.62:39831
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:58:52.525628 4330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:58:52.527082 4330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:58:52.539798 4335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:52.539721 4338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:52.539775 4336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:58:52.540966 4330 server_base.cc:1034] running on GCE node
I20260620 22:58:52.541570 4330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:58:52.542475 4330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:58:52.543717 4330 hybrid_clock.cc:648] HybridClock initialized: now 1781996332543653 us; error 45 us; skew 500 ppm
I20260620 22:58:52.544055 4330 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:58:52 dist-test-slave-f63b krb5kdc[4074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996332, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:58:52.547412 4330 init.cc:377] Logged in from keytab as kudu/127.2.237.2@KRBTEST.COM (short username kudu)
I20260620 22:58:52.548803 4330 webserver.cc:460] Webserver started at http://127.2.237.2:42755/ using document root <none> and password file <none>
I20260620 22:58:52.549594 4330 fs_manager.cc:359] Metadata directory not provided
I20260620 22:58:52.549727 4330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:58:52.550047 4330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:58:52.552965 4330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-1/data/instance:
uuid: "3c6151ac8a714a2583c1310aa9a5fbed"
format_stamp: "Formatted at 2026-06-20 22:58:52 on dist-test-slave-f63b"
server_key: "c7e325605dd2c57609f8b77aeb8ab7e7"
server_key_iv: "6ae422d8c7c72a6466e48cf1d9baac0f"
server_key_version: "encryptionkey@0"
I20260620 22:58:52.553745 4330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance:
uuid: "3c6151ac8a714a2583c1310aa9a5fbed"
format_stamp: "Formatted at 2026-06-20 22:58:52 on dist-test-slave-f63b"
server_key: "c7e325605dd2c57609f8b77aeb8ab7e7"
server_key_iv: "6ae422d8c7c72a6466e48cf1d9baac0f"
server_key_version: "encryptionkey@0"
I20260620 22:58:52.558876 4330 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260620 22:58:52.562243 4345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:52.563534 4330 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 22:58:52.563697 4330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "3c6151ac8a714a2583c1310aa9a5fbed"
format_stamp: "Formatted at 2026-06-20 22:58:52 on dist-test-slave-f63b"
server_key: "c7e325605dd2c57609f8b77aeb8ab7e7"
server_key_iv: "6ae422d8c7c72a6466e48cf1d9baac0f"
server_key_version: "encryptionkey@0"
I20260620 22:58:52.563901 4330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:58:52.592433 4330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:58:52.596676 4330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:58:52.596980 4330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:58:52.599718 4330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:58:52.599850 4330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:52.600049 4330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:58:52.600127 4330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 22:58:52 dist-test-slave-f63b krb5kdc[4074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996332, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:58:52.627481 4214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:52.600169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:60515 (local address 127.2.237.62:39831)
0620 22:58:52.600400 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:52.600410 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:52.601940 (+ 1530us) server_negotiation.cc:407] Connection header received
0620 22:58:52.604225 (+ 2285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:52.604233 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:52.604362 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:52.604468 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:52.607478 (+ 3010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:52.608481 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:52.611805 (+ 3324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:52.612408 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:52.612555 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:52.617890 (+ 5335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:52.617898 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:52.617905 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:52.617953 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:52.621821 (+ 3868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:52.622677 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:52.622683 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:52.622686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:52.622873 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:52.623598 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:52.623606 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:52.623609 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:52.624064 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:52.624229 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:52.626789 (+ 2560us) server_negotiation.cc:299] Negotiation successful
0620 22:58:52.627093 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260620 22:58:52.628540 4354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:52.600833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:39831 (local address 127.2.237.2:60515)
0620 22:58:52.601753 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:52.601804 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:52.603909 (+ 2105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:52.604801 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:52.604819 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:52.605820 (+ 1001us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:58:52.607198 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:52.607246 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:52.608667 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:52.608675 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:52.611592 (+ 2917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:52.611616 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:52.612925 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:52.612934 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:52.613237 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:52.614224 (+ 987us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:58:52.614259 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:58:52.617636 (+ 3377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:58:52.622019 (+ 4383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:52.622031 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:52.622049 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:52.622506 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:52.623005 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:52.623013 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:52.623019 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:52.623403 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:52.624324 (+ 921us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:58:52.624334 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:58:52.624948 (+ 614us) client_negotiation.cc:769] Sending connection context
0620 22:58:52.625335 (+ 387us) client_negotiation.cc:240] Negotiation successful
0620 22:58:52.626083 (+ 748us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":386,"threads_started":1}
I20260620 22:58:52.646937 4330 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:39403
I20260620 22:58:52.646981 4464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:39403 every 8 connection(s)
I20260620 22:58:52.648569 4330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 22:58:52.648926 4214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:52.634786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:59807 (local address 127.2.237.62:39831)
0620 22:58:52.634962 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:52.634966 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:52.635617 (+ 651us) server_negotiation.cc:407] Connection header received
0620 22:58:52.635809 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:52.635816 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:52.635879 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:52.635986 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:52.636891 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:52.637629 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:52.639388 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:52.640332 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:52.640511 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:52.642168 (+ 1657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:52.642177 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:52.642185 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:52.642229 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:52.645985 (+ 3756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:52.647076 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:52.647082 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:52.647085 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:52.647240 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:52.647600 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:52.647604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:52.647606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:52.647903 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:52.648012 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:52.648483 (+ 471us) server_negotiation.cc:299] Negotiation successful
0620 22:58:52.648684 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 22:58:52.649420 4449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:52.634885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:39831 (local address 127.2.237.2:59807)
0620 22:58:52.635518 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:52.635534 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:52.635682 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:52.636016 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:52.636021 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:52.636488 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:58:52.636756 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:52.636765 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:52.637755 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:52.637763 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:52.639181 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:52.639195 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:52.640539 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:52.640544 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:52.640652 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:52.641267 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:58:52.641284 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:58:52.641966 (+ 682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:58:52.646192 (+ 4226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:52.646198 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:52.646204 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:52.646887 (+ 683us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:52.647337 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:52.647341 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:52.647344 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:52.647487 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:52.648023 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:58:52.648031 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:58:52.648241 (+ 210us) client_negotiation.cc:769] Sending connection context
0620 22:58:52.648378 (+ 137us) client_negotiation.cc:240] Negotiation successful
0620 22:58:52.648853 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":361,"threads_started":1}
I20260620 22:58:52.654672 4465 heartbeater.cc:347] Connected to a master server at 127.2.237.62:39831
I20260620 22:58:52.654857 4465 heartbeater.cc:464] Registering TS with master...
I20260620 22:58:52.655426 4465 heartbeater.cc:511] Master 127.2.237.62:39831 requested a full tablet report, sending...
I20260620 22:58:52.656546 4119 ts_manager.cc:194] Registered new tserver with Master: 3c6151ac8a714a2583c1310aa9a5fbed (127.2.237.2:39403)
I20260620 22:58:52.657325 4119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.2@KRBTEST.COM'} at 127.2.237.2:60515
I20260620 22:58:52.657923 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 4330
I20260620 22:58:52.658126 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance
I20260620 22:58:52.658677 2996 external_mini_cluster.cc:1428] Setting key edc90f4a77f8ef5c23d29d50c1a09dcd
WARNING: no policy specified for kudu/127.2.237.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
I20260620 22:58:52.697890 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:0
--local_ip_for_outbound_sockets=127.2.237.3
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:39831
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:36135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:58:53.016196 4469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:58:53.016464 4469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:58:53.016558 4469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:58:53.024614 4469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:58:53.024726 4469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:58:53.024811 4469 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 22:58:53.024889 4469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 22:58:53.034976 4469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:36135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.237.62:39831
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:58:53.036288 4469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:58:53.037611 4469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:58:53.050769 4474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:53.050832 4477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:53.050808 4475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:58:53.051640 4469 server_base.cc:1034] running on GCE node
I20260620 22:58:53.052208 4469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:58:53.053084 4469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:58:53.054355 4469 hybrid_clock.cc:648] HybridClock initialized: now 1781996333054273 us; error 56 us; skew 500 ppm
I20260620 22:58:53.054762 4469 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:58:53 dist-test-slave-f63b krb5kdc[4074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996333, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:58:53.057870 4469 init.cc:377] Logged in from keytab as kudu/127.2.237.3@KRBTEST.COM (short username kudu)
I20260620 22:58:53.059311 4469 webserver.cc:460] Webserver started at http://127.2.237.3:40627/ using document root <none> and password file <none>
I20260620 22:58:53.060035 4469 fs_manager.cc:359] Metadata directory not provided
I20260620 22:58:53.060156 4469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:58:53.060479 4469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:58:53.063604 4469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-2/data/instance:
uuid: "9d25123880f346369887bb834bc97880"
format_stamp: "Formatted at 2026-06-20 22:58:53 on dist-test-slave-f63b"
server_key: "9487b52cbd07f18d0b462e5889b35508"
server_key_iv: "5c3a8f35eb7ec88068ebac62572c6fe1"
server_key_version: "encryptionkey@0"
I20260620 22:58:53.064407 4469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance:
uuid: "9d25123880f346369887bb834bc97880"
format_stamp: "Formatted at 2026-06-20 22:58:53 on dist-test-slave-f63b"
server_key: "9487b52cbd07f18d0b462e5889b35508"
server_key_iv: "5c3a8f35eb7ec88068ebac62572c6fe1"
server_key_version: "encryptionkey@0"
I20260620 22:58:53.069774 4469 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260620 22:58:53.073290 4484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:53.074668 4469 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 22:58:53.074833 4469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "9d25123880f346369887bb834bc97880"
format_stamp: "Formatted at 2026-06-20 22:58:53 on dist-test-slave-f63b"
server_key: "9487b52cbd07f18d0b462e5889b35508"
server_key_iv: "5c3a8f35eb7ec88068ebac62572c6fe1"
server_key_version: "encryptionkey@0"
I20260620 22:58:53.075047 4469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:58:53.090873 4469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:58:53.094770 4469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:58:53.095055 4469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:58:53.097708 4469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:58:53.097811 4469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:53.097951 4469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:58:53.098021 4469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 22:58:53 dist-test-slave-f63b krb5kdc[4074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996333, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:58:53.121268 4214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:53.098435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:59165 (local address 127.2.237.62:39831)
0620 22:58:53.098670 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:53.098678 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:53.100063 (+ 1385us) server_negotiation.cc:407] Connection header received
0620 22:58:53.102349 (+ 2286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:53.102359 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:53.102473 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:53.102630 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:53.104837 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.105933 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.108395 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.108889 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.109033 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:53.114046 (+ 5013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:53.114053 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:53.114057 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:53.114101 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:53.117659 (+ 3558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:53.118912 (+ 1253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:53.118918 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:53.118922 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:53.119060 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:53.119501 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:53.119507 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:53.119510 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:53.119802 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:53.119950 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:53.120684 (+ 734us) server_negotiation.cc:299] Negotiation successful
0620 22:58:53.120908 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260620 22:58:53.122519 4493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:53.099049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:39831 (local address 127.2.237.3:59165)
0620 22:58:53.099899 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:53.099939 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:53.101945 (+ 2006us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:53.102905 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:53.102914 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:53.103664 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:58:53.104645 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:53.104675 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.106105 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.106114 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:53.108239 (+ 2125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:53.108251 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.109179 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.109191 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:53.109558 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:53.110516 (+ 958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:58:53.110557 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:58:53.113795 (+ 3238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:58:53.117845 (+ 4050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:53.117860 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:53.117878 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:53.118722 (+ 844us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:53.119172 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:53.119177 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:53.119181 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:53.119373 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:53.119924 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:58:53.119932 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:58:53.120279 (+ 347us) client_negotiation.cc:769] Sending connection context
0620 22:58:53.120546 (+ 267us) client_negotiation.cc:240] Negotiation successful
0620 22:58:53.121079 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":379,"threads_started":1}
I20260620 22:58:53.143806 4214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:53.130565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:36945 (local address 127.2.237.62:39831)
0620 22:58:53.130797 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:53.130806 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:53.131471 (+ 665us) server_negotiation.cc:407] Connection header received
0620 22:58:53.131709 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:53.131717 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:53.131816 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:53.131934 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:53.133370 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.134049 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.135730 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.136348 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.136485 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:53.138323 (+ 1838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:53.138329 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:53.138334 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:53.138377 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:53.141219 (+ 2842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:53.141858 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:53.141865 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:53.141869 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:53.142007 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:53.142350 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:53.142355 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:53.142359 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:53.142689 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:53.142831 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:53.143241 (+ 410us) server_negotiation.cc:299] Negotiation successful
0620 22:58:53.143455 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 22:58:53.143908 4574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:53.130564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:39831 (local address 127.2.237.3:36945)
0620 22:58:53.131289 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:53.131367 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:53.131534 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:53.131969 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:53.131973 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:53.132743 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:58:53.133225 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:53.133239 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.134200 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.134208 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:53.135589 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:53.135609 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.136588 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.136595 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:53.136745 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:53.137464 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:58:53.137481 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:58:53.138152 (+ 671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:58:53.141351 (+ 3199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:53.141358 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:53.141362 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:53.141740 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:53.142115 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:53.142121 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:53.142124 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:53.142247 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:53.142809 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:58:53.142814 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:58:53.142992 (+ 178us) client_negotiation.cc:769] Sending connection context
0620 22:58:53.143145 (+ 153us) client_negotiation.cc:240] Negotiation successful
0620 22:58:53.143481 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":400,"threads_started":1}
I20260620 22:58:53.154628 4325 heartbeater.cc:503] Master 127.2.237.62:39831 was elected leader, sending a full tablet report...
I20260620 22:58:53.154690 4469 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:36923
I20260620 22:58:53.154640 4603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:36923 every 8 connection(s)
I20260620 22:58:53.156062 4469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 22:58:53.157774 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 4469
I20260620 22:58:53.157948 4604 heartbeater.cc:347] Connected to a master server at 127.2.237.62:39831
I20260620 22:58:53.158000 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance
I20260620 22:58:53.158174 4604 heartbeater.cc:464] Registering TS with master...
I20260620 22:58:53.158531 2996 external_mini_cluster.cc:1428] Setting key bead9f06972ddba7216c0472a3997f22
I20260620 22:58:53.159000 4604 heartbeater.cc:511] Master 127.2.237.62:39831 requested a full tablet report, sending...
I20260620 22:58:53.160542 4119 ts_manager.cc:194] Registered new tserver with Master: 9d25123880f346369887bb834bc97880 (127.2.237.3:36923)
I20260620 22:58:53.161780 4119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.3@KRBTEST.COM'} at 127.2.237.3:59165
I20260620 22:58:53.161967 2996 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 22:58:53 dist-test-slave-f63b krb5kdc[4074](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996333, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 22:58:53 dist-test-slave-f63b krb5kdc[4074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996333, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:58:53.208580 4214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:53.195020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43456 (local address 127.2.237.62:39831)
0620 22:58:53.195191 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:53.195195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:53.195505 (+ 310us) server_negotiation.cc:407] Connection header received
0620 22:58:53.195654 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:53.195658 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:53.195720 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:53.195812 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:53.197103 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.197859 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.199231 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.199632 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.199723 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:53.203519 (+ 3796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:53.203524 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:53.203527 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:53.203560 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:53.205922 (+ 2362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:53.206534 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:53.206544 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:53.206547 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:53.206741 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:53.207117 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:53.207123 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:53.207126 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:53.207404 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:53.207652 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:53.207956 (+ 304us) server_negotiation.cc:299] Negotiation successful
0620 22:58:53.208176 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260620 22:58:53.212510 4119 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43456:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 22:58:53.219256 4119 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 22:58:53.244246 4621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:53.235568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:42623 (local address 127.0.0.1:58328)
0620 22:58:53.236133 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:53.236149 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:53.236255 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:53.237279 (+ 1024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:53.237285 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:53.237313 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:58:53.237724 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:53.237734 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.239477 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.239487 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:53.241362 (+ 1875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:53.241376 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.243237 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.243243 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:53.243425 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:53.243490 (+ 65us) client_negotiation.cc:769] Sending connection context
0620 22:58:53.243610 (+ 120us) client_negotiation.cc:240] Negotiation successful
0620 22:58:53.243807 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"mutex_wait_us":93,"thread_start_us":310,"threads_started":1}
I20260620 22:58:53.244395 4618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:53.231170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:36923 (local address 127.0.0.1:53474)
0620 22:58:53.232113 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:53.232154 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:53.232311 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:53.237112 (+ 4801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:53.237117 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:53.237140 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:58:53.237670 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:53.237688 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.239594 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.239602 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:53.241784 (+ 2182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:53.241795 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.243543 (+ 1748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.243553 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:53.243772 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:53.243805 (+ 33us) client_negotiation.cc:769] Sending connection context
0620 22:58:53.243885 (+ 80us) client_negotiation.cc:240] Negotiation successful
0620 22:58:53.244012 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"mutex_wait_us":10,"thread_start_us":4005,"threads_started":1}
I20260620 22:58:53.244467 4620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:53.231714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55196 (local address 127.2.237.2:39403)
0620 22:58:53.235017 (+ 3303us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:53.235028 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:53.235571 (+ 543us) server_negotiation.cc:407] Connection header received
0620 22:58:53.235890 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:53.235899 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:53.236205 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:53.236413 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:58:53.237322 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.238672 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.241593 (+ 2921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.243032 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.243167 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:53.243392 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:53.243988 (+ 596us) server_negotiation.cc:299] Negotiation successful
0620 22:58:53.244170 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3159,"thread_start_us":3058,"threads_started":1}
I20260620 22:58:53.244535 4622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:53.235813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58328 (local address 127.2.237.1:42623)
0620 22:58:53.236698 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:53.236705 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:53.236733 (+ 28us) server_negotiation.cc:407] Connection header received
0620 22:58:53.236827 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:53.236847 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:53.237066 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:53.237920 (+ 854us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:58:53.238070 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.239312 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.241510 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.243032 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.243794 (+ 762us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:53.244011 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:53.244076 (+ 65us) server_negotiation.cc:299] Negotiation successful
0620 22:58:53.244237 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":747,"thread_start_us":627,"threads_started":1}
I20260620 22:58:53.245128 4619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:53.231442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53474 (local address 127.2.237.3:36923)
0620 22:58:53.235467 (+ 4025us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:53.235477 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:53.235515 (+ 38us) server_negotiation.cc:407] Connection header received
0620 22:58:53.236662 (+ 1147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:53.236670 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:53.236893 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:53.237106 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:58:53.237907 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.239441 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.241959 (+ 2518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.243283 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.243448 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:53.243683 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:53.244618 (+ 935us) server_negotiation.cc:299] Negotiation successful
0620 22:58:53.244793 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3866,"thread_start_us":3679,"threads_started":1}
I20260620 22:58:53.246274 4617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:53.231378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:39403 (local address 127.0.0.1:55196)
0620 22:58:53.235461 (+ 4083us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:53.235484 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:53.235675 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:53.236694 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:53.236701 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:53.236729 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:58:53.237107 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:53.237120 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.238814 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.238820 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:53.241392 (+ 2572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:53.241407 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.243570 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.243579 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:53.243780 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:53.243810 (+ 30us) client_negotiation.cc:769] Sending connection context
0620 22:58:53.245775 (+ 1965us) client_negotiation.cc:240] Negotiation successful
0620 22:58:53.245906 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3936,"mutex_wait_us":85,"thread_start_us":479,"threads_started":1}
I20260620 22:58:53.248600 4256 tablet_service.cc:1467] Processing CreateTablet for tablet 43227a7cad924b27a8429c621afd10dc (DEFAULT_TABLE table=test-table [id=54716d7473ab4aa1bb34ebe172f744c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:58:53.248901 4534 tablet_service.cc:1467] Processing CreateTablet for tablet 43227a7cad924b27a8429c621afd10dc (DEFAULT_TABLE table=test-table [id=54716d7473ab4aa1bb34ebe172f744c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:58:53.249879 4256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43227a7cad924b27a8429c621afd10dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:58:53.249562 4395 tablet_service.cc:1467] Processing CreateTablet for tablet 43227a7cad924b27a8429c621afd10dc (DEFAULT_TABLE table=test-table [id=54716d7473ab4aa1bb34ebe172f744c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:58:53.250273 4534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43227a7cad924b27a8429c621afd10dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:58:53.250777 4395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43227a7cad924b27a8429c621afd10dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:58:53.261214 4623 tablet_bootstrap.cc:492] T 43227a7cad924b27a8429c621afd10dc P 6d958c8a03d94635a6b46cceff8dbfe8: Bootstrap starting.
I20260620 22:58:53.264550 4624 tablet_bootstrap.cc:492] T 43227a7cad924b27a8429c621afd10dc P 9d25123880f346369887bb834bc97880: Bootstrap starting.
I20260620 22:58:53.265053 4623 tablet_bootstrap.cc:654] T 43227a7cad924b27a8429c621afd10dc P 6d958c8a03d94635a6b46cceff8dbfe8: Neither blocks nor log segments found. Creating new log.
I20260620 22:58:53.266070 4625 tablet_bootstrap.cc:492] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed: Bootstrap starting.
I20260620 22:58:53.266872 4623 log.cc:824] T 43227a7cad924b27a8429c621afd10dc P 6d958c8a03d94635a6b46cceff8dbfe8: Log is configured to *not* fsync() on all Append() calls
I20260620 22:58:53.269538 4625 tablet_bootstrap.cc:654] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed: Neither blocks nor log segments found. Creating new log.
I20260620 22:58:53.269747 4624 tablet_bootstrap.cc:654] T 43227a7cad924b27a8429c621afd10dc P 9d25123880f346369887bb834bc97880: Neither blocks nor log segments found. Creating new log.
I20260620 22:58:53.270278 4623 tablet_bootstrap.cc:492] T 43227a7cad924b27a8429c621afd10dc P 6d958c8a03d94635a6b46cceff8dbfe8: No bootstrap required, opened a new log
I20260620 22:58:53.270555 4623 ts_tablet_manager.cc:1397] T 43227a7cad924b27a8429c621afd10dc P 6d958c8a03d94635a6b46cceff8dbfe8: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260620 22:58:53.270747 4625 log.cc:824] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed: Log is configured to *not* fsync() on all Append() calls
I20260620 22:58:53.270857 4624 log.cc:824] T 43227a7cad924b27a8429c621afd10dc P 9d25123880f346369887bb834bc97880: Log is configured to *not* fsync() on all Append() calls
I20260620 22:58:53.273732 4625 tablet_bootstrap.cc:492] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed: No bootstrap required, opened a new log
I20260620 22:58:53.274034 4625 ts_tablet_manager.cc:1397] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260620 22:58:53.274430 4624 tablet_bootstrap.cc:492] T 43227a7cad924b27a8429c621afd10dc P 9d25123880f346369887bb834bc97880: No bootstrap required, opened a new log
I20260620 22:58:53.274703 4624 ts_tablet_manager.cc:1397] T 43227a7cad924b27a8429c621afd10dc P 9d25123880f346369887bb834bc97880: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.005s
I20260620 22:58:53.279873 4623 raft_consensus.cc:348] T 43227a7cad924b27a8429c621afd10dc P 6d958c8a03d94635a6b46cceff8dbfe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } }
I20260620 22:58:53.280288 4623 raft_consensus.cc:374] T 43227a7cad924b27a8429c621afd10dc P 6d958c8a03d94635a6b46cceff8dbfe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:58:53.280378 4623 raft_consensus.cc:729] T 43227a7cad924b27a8429c621afd10dc P 6d958c8a03d94635a6b46cceff8dbfe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d958c8a03d94635a6b46cceff8dbfe8, State: Initialized, Role: FOLLOWER
I20260620 22:58:53.280990 4623 consensus_queue.cc:260] T 43227a7cad924b27a8429c621afd10dc P 6d958c8a03d94635a6b46cceff8dbfe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } }
I20260620 22:58:53.283995 4623 ts_tablet_manager.cc:1428] T 43227a7cad924b27a8429c621afd10dc P 6d958c8a03d94635a6b46cceff8dbfe8: Time spent starting tablet: real 0.013s user 0.010s sys 0.002s
I20260620 22:58:53.285518 4625 raft_consensus.cc:348] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } }
I20260620 22:58:53.285483 4624 raft_consensus.cc:348] T 43227a7cad924b27a8429c621afd10dc P 9d25123880f346369887bb834bc97880 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } }
I20260620 22:58:53.285874 4625 raft_consensus.cc:374] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:58:53.285913 4624 raft_consensus.cc:374] T 43227a7cad924b27a8429c621afd10dc P 9d25123880f346369887bb834bc97880 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:58:53.286000 4624 raft_consensus.cc:729] T 43227a7cad924b27a8429c621afd10dc P 9d25123880f346369887bb834bc97880 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d25123880f346369887bb834bc97880, State: Initialized, Role: FOLLOWER
I20260620 22:58:53.286001 4625 raft_consensus.cc:729] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c6151ac8a714a2583c1310aa9a5fbed, State: Initialized, Role: FOLLOWER
I20260620 22:58:53.286557 4625 consensus_queue.cc:260] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } }
I20260620 22:58:53.286695 4624 consensus_queue.cc:260] T 43227a7cad924b27a8429c621afd10dc P 9d25123880f346369887bb834bc97880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } }
I20260620 22:58:53.287678 4465 heartbeater.cc:503] Master 127.2.237.62:39831 was elected leader, sending a full tablet report...
I20260620 22:58:53.288327 4625 ts_tablet_manager.cc:1428] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed: Time spent starting tablet: real 0.014s user 0.007s sys 0.007s
I20260620 22:58:53.291288 4604 heartbeater.cc:503] Master 127.2.237.62:39831 was elected leader, sending a full tablet report...
I20260620 22:58:53.291795 4624 ts_tablet_manager.cc:1428] T 43227a7cad924b27a8429c621afd10dc P 9d25123880f346369887bb834bc97880: Time spent starting tablet: real 0.017s user 0.009s sys 0.006s
I20260620 22:58:53.341913 4630 raft_consensus.cc:482] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 22:58:53.342607 4630 raft_consensus.cc:504] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } }
I20260620 22:58:53.344259 4630 leader_election.cc:288] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d25123880f346369887bb834bc97880 (127.2.237.3:36923), 6d958c8a03d94635a6b46cceff8dbfe8 (127.2.237.1:42623)
I20260620 22:58:53.351367 4619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:53.344621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:58507 (local address 127.2.237.3:36923)
0620 22:58:53.344838 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:53.344846 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:53.345468 (+ 622us) server_negotiation.cc:407] Connection header received
0620 22:58:53.345695 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:53.345702 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:53.345806 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:53.345860 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:58:53.346621 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.347487 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.349488 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.350363 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.350470 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:53.350576 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:53.350965 (+ 389us) server_negotiation.cc:299] Negotiation successful
0620 22:58:53.351100 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 22:58:53.351377 4622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:53.345566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:35169 (local address 127.2.237.1:42623)
0620 22:58:53.345749 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:53.345754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:53.346104 (+ 350us) server_negotiation.cc:407] Connection header received
0620 22:58:53.346235 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:53.346242 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:53.346338 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:53.346429 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:58:53.347070 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.347916 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.349681 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.350463 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.350620 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:53.350722 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:53.351023 (+ 301us) server_negotiation.cc:299] Negotiation successful
0620 22:58:53.351142 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260620 22:58:53.351799 4633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:53.345399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:42623 (local address 127.2.237.2:35169)
0620 22:58:53.345997 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:53.346010 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:53.346115 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:53.346455 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:53.346458 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:53.346474 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:58:53.346902 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:53.346914 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.348044 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.348048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:53.349533 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:53.349548 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.350716 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.350720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:53.350839 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:53.350861 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 22:58:53.350980 (+ 119us) client_negotiation.cc:240] Negotiation successful
0620 22:58:53.351110 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":2,"thread_start_us":297,"threads_started":1}
I20260620 22:58:53.351925 4632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:53.344634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:36923 (local address 127.2.237.2:58507)
0620 22:58:53.345307 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:53.345353 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:53.345518 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:53.346019 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:53.346022 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:53.346043 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:58:53.346433 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:53.346445 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.347620 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.347626 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:53.349321 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:53.349332 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.350604 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.350610 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:53.350803 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:53.350826 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 22:58:53.350899 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 22:58:53.351051 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":58,"thread_start_us":349,"threads_started":1}
I20260620 22:58:53.352896 4276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43227a7cad924b27a8429c621afd10dc" candidate_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" is_pre_election: true
I20260620 22:58:53.352898 4554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43227a7cad924b27a8429c621afd10dc" candidate_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d25123880f346369887bb834bc97880" is_pre_election: true
I20260620 22:58:53.353904 4276 raft_consensus.cc:2454] T 43227a7cad924b27a8429c621afd10dc P 6d958c8a03d94635a6b46cceff8dbfe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c6151ac8a714a2583c1310aa9a5fbed in term 0.
I20260620 22:58:53.353941 4554 raft_consensus.cc:2454] T 43227a7cad924b27a8429c621afd10dc P 9d25123880f346369887bb834bc97880 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c6151ac8a714a2583c1310aa9a5fbed in term 0.
I20260620 22:58:53.354547 4346 leader_election.cc:302] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c6151ac8a714a2583c1310aa9a5fbed, 6d958c8a03d94635a6b46cceff8dbfe8; no voters:
I20260620 22:58:53.354976 4630 raft_consensus.cc:2781] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 22:58:53.355120 4630 raft_consensus.cc:482] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 22:58:53.355219 4630 raft_consensus.cc:3037] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:58:53.357446 4630 raft_consensus.cc:504] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } }
I20260620 22:58:53.358089 4630 leader_election.cc:288] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed [CANDIDATE]: Term 1 election: Requested vote from peers 9d25123880f346369887bb834bc97880 (127.2.237.3:36923), 6d958c8a03d94635a6b46cceff8dbfe8 (127.2.237.1:42623)
I20260620 22:58:53.358484 4554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43227a7cad924b27a8429c621afd10dc" candidate_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d25123880f346369887bb834bc97880"
I20260620 22:58:53.358657 4276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43227a7cad924b27a8429c621afd10dc" candidate_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d958c8a03d94635a6b46cceff8dbfe8"
I20260620 22:58:53.358727 4554 raft_consensus.cc:3037] T 43227a7cad924b27a8429c621afd10dc P 9d25123880f346369887bb834bc97880 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:58:53.358873 4276 raft_consensus.cc:3037] T 43227a7cad924b27a8429c621afd10dc P 6d958c8a03d94635a6b46cceff8dbfe8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:58:53.360917 4554 raft_consensus.cc:2454] T 43227a7cad924b27a8429c621afd10dc P 9d25123880f346369887bb834bc97880 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c6151ac8a714a2583c1310aa9a5fbed in term 1.
I20260620 22:58:53.360950 4276 raft_consensus.cc:2454] T 43227a7cad924b27a8429c621afd10dc P 6d958c8a03d94635a6b46cceff8dbfe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c6151ac8a714a2583c1310aa9a5fbed in term 1.
I20260620 22:58:53.361408 4346 leader_election.cc:302] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c6151ac8a714a2583c1310aa9a5fbed, 9d25123880f346369887bb834bc97880; no voters:
I20260620 22:58:53.361665 4630 raft_consensus.cc:2781] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:58:53.362145 4630 raft_consensus.cc:686] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed [term 1 LEADER]: Becoming Leader. State: Replica: 3c6151ac8a714a2583c1310aa9a5fbed, State: Running, Role: LEADER
I20260620 22:58:53.362731 4630 consensus_queue.cc:237] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } }
I20260620 22:58:53.367213 4118 catalog_manager.cc:5495] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c6151ac8a714a2583c1310aa9a5fbed (127.2.237.2). New cstate: current_term: 1 leader_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } health_report { overall_health: UNKNOWN } } }
W20260620 22:58:53.394057 4327 tablet.cc:2369] T 43227a7cad924b27a8429c621afd10dc P 6d958c8a03d94635a6b46cceff8dbfe8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 22:58:53.404871 4466 tablet.cc:2369] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 22:58:53.405872 4620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:53.400386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55206 (local address 127.2.237.2:39403)
0620 22:58:53.400568 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:53.400572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:53.400639 (+ 67us) server_negotiation.cc:407] Connection header received
0620 22:58:53.400898 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:53.400901 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:53.400986 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:53.401110 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 22:58:53.401404 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.402137 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.403463 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.403925 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.404037 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:53.404466 (+ 429us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 22:58:53.405249 (+ 783us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 22:58:53.405402 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:53.405511 (+ 109us) server_negotiation.cc:299] Negotiation successful
0620 22:58:53.405603 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260620 22:58:53.407636 4605 tablet.cc:2369] T 43227a7cad924b27a8429c621afd10dc P 9d25123880f346369887bb834bc97880: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 22:58:53.413906 4554 raft_consensus.cc:1261] T 43227a7cad924b27a8429c621afd10dc P 9d25123880f346369887bb834bc97880 [term 1 FOLLOWER]: Refusing update from remote peer 3c6151ac8a714a2583c1310aa9a5fbed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:58:53.414033 4276 raft_consensus.cc:1261] T 43227a7cad924b27a8429c621afd10dc P 6d958c8a03d94635a6b46cceff8dbfe8 [term 1 FOLLOWER]: Refusing update from remote peer 3c6151ac8a714a2583c1310aa9a5fbed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:58:53.414935 4630 consensus_queue.cc:1035] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:58:53.415268 4634 consensus_queue.cc:1035] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:58:53.434664 4638 mvcc.cc:205] Tried to move back new op lower bound from 7299056981653344256 to 7299056981458190336. Current Snapshot: MvccSnapshot[applied={T|T < 7299056981653344256}]
I20260620 22:58:53.452310 4119 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 22:58:53.452600 4119 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 22:58:53.456393 4119 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2e2c41911b804b5ea76ef0485b087abd: Sending DeleteTablet for 3 replicas of tablet 43227a7cad924b27a8429c621afd10dc
I20260620 22:58:53.457527 4395 tablet_service.cc:1514] Processing DeleteTablet for tablet 43227a7cad924b27a8429c621afd10dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:58:53 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:55196
I20260620 22:58:53.457975 4534 tablet_service.cc:1514] Processing DeleteTablet for tablet 43227a7cad924b27a8429c621afd10dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:58:53 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:53474
I20260620 22:58:53.458705 4256 tablet_service.cc:1514] Processing DeleteTablet for tablet 43227a7cad924b27a8429c621afd10dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:58:53 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:58328
I20260620 22:58:53.458811 4119 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43456:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 22:58:53.459046 4647 tablet_replica.cc:330] T 43227a7cad924b27a8429c621afd10dc P 9d25123880f346369887bb834bc97880: stopping tablet replica
W20260620 22:58:53.459452 4119 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 22:58:53.459452 4647 raft_consensus.cc:2229] T 43227a7cad924b27a8429c621afd10dc P 9d25123880f346369887bb834bc97880 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 22:58:53.459914 4647 raft_consensus.cc:2258] T 43227a7cad924b27a8429c621afd10dc P 9d25123880f346369887bb834bc97880 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:58:53.462227 4647 ts_tablet_manager.cc:1905] T 43227a7cad924b27a8429c621afd10dc P 9d25123880f346369887bb834bc97880: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:58:53.468317 4395 tablet_service.cc:1467] Processing CreateTablet for tablet 937a808ac7ee43ffbaf2be5dd1a34ac7 (DEFAULT_TABLE table=test-table [id=44410520bc044d1589ff11b81e64e305]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:58:53.468851 4395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 937a808ac7ee43ffbaf2be5dd1a34ac7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:58:53.469054 4647 ts_tablet_manager.cc:1918] T 43227a7cad924b27a8429c621afd10dc P 9d25123880f346369887bb834bc97880: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 22:58:53.469192 4647 log.cc:1196] T 43227a7cad924b27a8429c621afd10dc P 9d25123880f346369887bb834bc97880: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-2/wal/wals/43227a7cad924b27a8429c621afd10dc
I20260620 22:58:53.469702 4647 ts_tablet_manager.cc:1939] T 43227a7cad924b27a8429c621afd10dc P 9d25123880f346369887bb834bc97880: Deleting consensus metadata
I20260620 22:58:53.471217 4648 tablet_replica.cc:330] T 43227a7cad924b27a8429c621afd10dc P 6d958c8a03d94635a6b46cceff8dbfe8: stopping tablet replica
I20260620 22:58:53.472056 4256 tablet_service.cc:1467] Processing CreateTablet for tablet 937a808ac7ee43ffbaf2be5dd1a34ac7 (DEFAULT_TABLE table=test-table [id=44410520bc044d1589ff11b81e64e305]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:58:53.472308 4534 tablet_service.cc:1467] Processing CreateTablet for tablet 937a808ac7ee43ffbaf2be5dd1a34ac7 (DEFAULT_TABLE table=test-table [id=44410520bc044d1589ff11b81e64e305]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:58:53.472491 4256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 937a808ac7ee43ffbaf2be5dd1a34ac7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:58:53.472702 4534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 937a808ac7ee43ffbaf2be5dd1a34ac7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:58:53.477339 4625 tablet_bootstrap.cc:492] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed: Bootstrap starting.
I20260620 22:58:53.477460 4646 tablet_replica.cc:330] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed: stopping tablet replica
I20260620 22:58:53.477905 4646 raft_consensus.cc:2229] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed [term 1 LEADER]: Raft consensus shutting down.
I20260620 22:58:53.479167 4648 raft_consensus.cc:2229] T 43227a7cad924b27a8429c621afd10dc P 6d958c8a03d94635a6b46cceff8dbfe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 22:58:53.479344 4623 tablet_bootstrap.cc:492] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 6d958c8a03d94635a6b46cceff8dbfe8: Bootstrap starting.
I20260620 22:58:53.479346 4646 raft_consensus.cc:2258] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:58:53.479691 4648 raft_consensus.cc:2258] T 43227a7cad924b27a8429c621afd10dc P 6d958c8a03d94635a6b46cceff8dbfe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:58:53.480609 4103 catalog_manager.cc:4841] TS 9d25123880f346369887bb834bc97880 (127.2.237.3:36923): tablet 43227a7cad924b27a8429c621afd10dc (table test-table [id=54716d7473ab4aa1bb34ebe172f744c1]) successfully deleted
I20260620 22:58:53.481032 4623 tablet_bootstrap.cc:654] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 6d958c8a03d94635a6b46cceff8dbfe8: Neither blocks nor log segments found. Creating new log.
I20260620 22:58:53.481739 4625 tablet_bootstrap.cc:654] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed: Neither blocks nor log segments found. Creating new log.
I20260620 22:58:53.481918 4646 ts_tablet_manager.cc:1905] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:58:53.484798 4624 tablet_bootstrap.cc:492] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 9d25123880f346369887bb834bc97880: Bootstrap starting.
I20260620 22:58:53.487530 4624 tablet_bootstrap.cc:654] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 9d25123880f346369887bb834bc97880: Neither blocks nor log segments found. Creating new log.
I20260620 22:58:53.489795 4646 ts_tablet_manager.cc:1918] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 22:58:53.489964 4646 log.cc:1196] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-1/wal/wals/43227a7cad924b27a8429c621afd10dc
I20260620 22:58:53.490510 4646 ts_tablet_manager.cc:1939] T 43227a7cad924b27a8429c621afd10dc P 3c6151ac8a714a2583c1310aa9a5fbed: Deleting consensus metadata
I20260620 22:58:53.492499 4648 ts_tablet_manager.cc:1905] T 43227a7cad924b27a8429c621afd10dc P 6d958c8a03d94635a6b46cceff8dbfe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:58:53.495854 4625 tablet_bootstrap.cc:492] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed: No bootstrap required, opened a new log
I20260620 22:58:53.496080 4625 ts_tablet_manager.cc:1397] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.003s
I20260620 22:58:53.496976 4625 raft_consensus.cc:348] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } }
I20260620 22:58:53.497224 4625 raft_consensus.cc:374] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:58:53.497288 4624 tablet_bootstrap.cc:492] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 9d25123880f346369887bb834bc97880: No bootstrap required, opened a new log
I20260620 22:58:53.497308 4625 raft_consensus.cc:729] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c6151ac8a714a2583c1310aa9a5fbed, State: Initialized, Role: FOLLOWER
I20260620 22:58:53.497468 4624 ts_tablet_manager.cc:1397] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 9d25123880f346369887bb834bc97880: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.002s
I20260620 22:58:53.497565 4625 consensus_queue.cc:260] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } }
I20260620 22:58:53.498107 4625 ts_tablet_manager.cc:1428] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260620 22:58:53.498242 4624 raft_consensus.cc:348] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 9d25123880f346369887bb834bc97880 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } }
I20260620 22:58:53.498476 4624 raft_consensus.cc:374] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 9d25123880f346369887bb834bc97880 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:58:53.498592 4624 raft_consensus.cc:729] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 9d25123880f346369887bb834bc97880 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d25123880f346369887bb834bc97880, State: Initialized, Role: FOLLOWER
I20260620 22:58:53.499361 4648 ts_tablet_manager.cc:1918] T 43227a7cad924b27a8429c621afd10dc P 6d958c8a03d94635a6b46cceff8dbfe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 22:58:53.499509 4648 log.cc:1196] T 43227a7cad924b27a8429c621afd10dc P 6d958c8a03d94635a6b46cceff8dbfe8: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-0/wal/wals/43227a7cad924b27a8429c621afd10dc
I20260620 22:58:53.500483 4648 ts_tablet_manager.cc:1939] T 43227a7cad924b27a8429c621afd10dc P 6d958c8a03d94635a6b46cceff8dbfe8: Deleting consensus metadata
I20260620 22:58:53.501005 4105 catalog_manager.cc:4841] TS 3c6151ac8a714a2583c1310aa9a5fbed (127.2.237.2:39403): tablet 43227a7cad924b27a8429c621afd10dc (table test-table [id=54716d7473ab4aa1bb34ebe172f744c1]) successfully deleted
I20260620 22:58:53.502166 4103 catalog_manager.cc:4841] TS 6d958c8a03d94635a6b46cceff8dbfe8 (127.2.237.1:42623): tablet 43227a7cad924b27a8429c621afd10dc (table test-table [id=54716d7473ab4aa1bb34ebe172f744c1]) successfully deleted
I20260620 22:58:53.502352 4623 tablet_bootstrap.cc:492] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 6d958c8a03d94635a6b46cceff8dbfe8: No bootstrap required, opened a new log
I20260620 22:58:53.502188 4624 consensus_queue.cc:260] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 9d25123880f346369887bb834bc97880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } }
I20260620 22:58:53.502480 4623 ts_tablet_manager.cc:1397] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 6d958c8a03d94635a6b46cceff8dbfe8: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.004s
I20260620 22:58:53.503237 4623 raft_consensus.cc:348] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 6d958c8a03d94635a6b46cceff8dbfe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } }
I20260620 22:58:53.503381 4624 ts_tablet_manager.cc:1428] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 9d25123880f346369887bb834bc97880: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260620 22:58:53.503456 4623 raft_consensus.cc:374] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 6d958c8a03d94635a6b46cceff8dbfe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:58:53.503536 4623 raft_consensus.cc:729] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 6d958c8a03d94635a6b46cceff8dbfe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d958c8a03d94635a6b46cceff8dbfe8, State: Initialized, Role: FOLLOWER
I20260620 22:58:53.503783 4623 consensus_queue.cc:260] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 6d958c8a03d94635a6b46cceff8dbfe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } }
I20260620 22:58:53.504411 4623 ts_tablet_manager.cc:1428] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 6d958c8a03d94635a6b46cceff8dbfe8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 22:58:53.568746 4630 raft_consensus.cc:482] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 22:58:53.568966 4630 raft_consensus.cc:504] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } }
I20260620 22:58:53.569710 4630 leader_election.cc:288] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d958c8a03d94635a6b46cceff8dbfe8 (127.2.237.1:42623), 9d25123880f346369887bb834bc97880 (127.2.237.3:36923)
I20260620 22:58:53.570209 4275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "937a808ac7ee43ffbaf2be5dd1a34ac7" candidate_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" is_pre_election: true
I20260620 22:58:53.570343 4554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "937a808ac7ee43ffbaf2be5dd1a34ac7" candidate_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d25123880f346369887bb834bc97880" is_pre_election: true
I20260620 22:58:53.570487 4275 raft_consensus.cc:2454] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 6d958c8a03d94635a6b46cceff8dbfe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c6151ac8a714a2583c1310aa9a5fbed in term 0.
I20260620 22:58:53.570628 4554 raft_consensus.cc:2454] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 9d25123880f346369887bb834bc97880 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c6151ac8a714a2583c1310aa9a5fbed in term 0.
I20260620 22:58:53.571074 4346 leader_election.cc:302] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c6151ac8a714a2583c1310aa9a5fbed, 9d25123880f346369887bb834bc97880; no voters:
I20260620 22:58:53.571372 4630 raft_consensus.cc:2781] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 22:58:53.571492 4630 raft_consensus.cc:482] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 22:58:53.571600 4630 raft_consensus.cc:3037] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:58:53.573809 4630 raft_consensus.cc:504] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } }
I20260620 22:58:53.574429 4630 leader_election.cc:288] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed [CANDIDATE]: Term 1 election: Requested vote from peers 6d958c8a03d94635a6b46cceff8dbfe8 (127.2.237.1:42623), 9d25123880f346369887bb834bc97880 (127.2.237.3:36923)
I20260620 22:58:53.574831 4275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "937a808ac7ee43ffbaf2be5dd1a34ac7" candidate_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d958c8a03d94635a6b46cceff8dbfe8"
I20260620 22:58:53.574939 4554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "937a808ac7ee43ffbaf2be5dd1a34ac7" candidate_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d25123880f346369887bb834bc97880"
I20260620 22:58:53.575038 4275 raft_consensus.cc:3037] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 6d958c8a03d94635a6b46cceff8dbfe8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:58:53.575138 4554 raft_consensus.cc:3037] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 9d25123880f346369887bb834bc97880 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:58:53.577068 4275 raft_consensus.cc:2454] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 6d958c8a03d94635a6b46cceff8dbfe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c6151ac8a714a2583c1310aa9a5fbed in term 1.
I20260620 22:58:53.577454 4346 leader_election.cc:302] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c6151ac8a714a2583c1310aa9a5fbed, 6d958c8a03d94635a6b46cceff8dbfe8; no voters:
I20260620 22:58:53.577739 4630 raft_consensus.cc:2781] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:58:53.577914 4630 raft_consensus.cc:686] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed [term 1 LEADER]: Becoming Leader. State: Replica: 3c6151ac8a714a2583c1310aa9a5fbed, State: Running, Role: LEADER
I20260620 22:58:53.578351 4554 raft_consensus.cc:2454] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 9d25123880f346369887bb834bc97880 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c6151ac8a714a2583c1310aa9a5fbed in term 1.
I20260620 22:58:53.578184 4630 consensus_queue.cc:237] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } }
I20260620 22:58:53.581429 4118 catalog_manager.cc:5495] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c6151ac8a714a2583c1310aa9a5fbed (127.2.237.2). New cstate: current_term: 1 leader_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } health_report { overall_health: UNKNOWN } } }
Jun 20 22:58:53 dist-test-slave-f63b krb5kdc[4074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996331, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:58:53.632642 4658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:53.617497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:39831 (local address 127.0.0.1:43462)
0620 22:58:53.618181 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:53.618198 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:53.618335 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:53.618693 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:53.618696 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:53.619383 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:58:53.619642 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:53.619650 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.621069 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.621073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:53.622037 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:53.622048 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.623069 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.623075 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:53.623271 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:53.623988 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:58:53.624005 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:58:53.626128 (+ 2123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:58:53.630153 (+ 4025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:53.630163 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:53.630168 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:53.630676 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:53.631053 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:53.631056 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:53.631058 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:53.631183 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:53.631792 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:58:53.631798 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:58:53.632014 (+ 216us) client_negotiation.cc:769] Sending connection context
0620 22:58:53.632148 (+ 134us) client_negotiation.cc:240] Negotiation successful
0620 22:58:53.632333 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":383,"threads_started":1}
I20260620 22:58:53.632881 4214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:53.617481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43462 (local address 127.2.237.62:39831)
0620 22:58:53.617699 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:53.617705 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:53.618292 (+ 587us) server_negotiation.cc:407] Connection header received
0620 22:58:53.618458 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:53.618464 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:53.618542 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:53.618678 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:53.619791 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.620940 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.622172 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.622874 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.623003 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:53.626291 (+ 3288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:53.626298 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:53.626303 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:53.626358 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:53.630020 (+ 3662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:53.630780 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:53.630803 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:53.630807 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:53.630944 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:53.631301 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:53.631306 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:53.631310 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:53.631675 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:53.631800 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:53.632284 (+ 484us) server_negotiation.cc:299] Negotiation successful
0620 22:58:53.632519 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260620 22:58:53.637966 4118 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:43462:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260620 22:58:53.639091 4118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 22:58:53.645892 4256 tablet_service.cc:1467] Processing CreateTablet for tablet 81d76a1d1f4842c685651cfb5661b741 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b4edd8cfea0b4794b40f3d5b0a825422]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 22:58:53.646474 4256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81d76a1d1f4842c685651cfb5661b741. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:58:53.646260 4395 tablet_service.cc:1467] Processing CreateTablet for tablet 81d76a1d1f4842c685651cfb5661b741 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b4edd8cfea0b4794b40f3d5b0a825422]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 22:58:53.646868 4395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81d76a1d1f4842c685651cfb5661b741. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:58:53.654352 4625 tablet_bootstrap.cc:492] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed: Bootstrap starting.
I20260620 22:58:53.655210 4534 tablet_service.cc:1467] Processing CreateTablet for tablet 81d76a1d1f4842c685651cfb5661b741 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b4edd8cfea0b4794b40f3d5b0a825422]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 22:58:53.656174 4534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81d76a1d1f4842c685651cfb5661b741. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:58:53.656646 4625 tablet_bootstrap.cc:654] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed: Neither blocks nor log segments found. Creating new log.
I20260620 22:58:53.665097 4623 tablet_bootstrap.cc:492] T 81d76a1d1f4842c685651cfb5661b741 P 6d958c8a03d94635a6b46cceff8dbfe8: Bootstrap starting.
I20260620 22:58:53.667436 4623 tablet_bootstrap.cc:654] T 81d76a1d1f4842c685651cfb5661b741 P 6d958c8a03d94635a6b46cceff8dbfe8: Neither blocks nor log segments found. Creating new log.
I20260620 22:58:53.669778 4624 tablet_bootstrap.cc:492] T 81d76a1d1f4842c685651cfb5661b741 P 9d25123880f346369887bb834bc97880: Bootstrap starting.
I20260620 22:58:53.672062 4624 tablet_bootstrap.cc:654] T 81d76a1d1f4842c685651cfb5661b741 P 9d25123880f346369887bb834bc97880: Neither blocks nor log segments found. Creating new log.
I20260620 22:58:53.674062 4623 tablet_bootstrap.cc:492] T 81d76a1d1f4842c685651cfb5661b741 P 6d958c8a03d94635a6b46cceff8dbfe8: No bootstrap required, opened a new log
I20260620 22:58:53.674239 4623 ts_tablet_manager.cc:1397] T 81d76a1d1f4842c685651cfb5661b741 P 6d958c8a03d94635a6b46cceff8dbfe8: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260620 22:58:53.675203 4623 raft_consensus.cc:348] T 81d76a1d1f4842c685651cfb5661b741 P 6d958c8a03d94635a6b46cceff8dbfe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } }
I20260620 22:58:53.675523 4623 raft_consensus.cc:374] T 81d76a1d1f4842c685651cfb5661b741 P 6d958c8a03d94635a6b46cceff8dbfe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:58:53.675608 4623 raft_consensus.cc:729] T 81d76a1d1f4842c685651cfb5661b741 P 6d958c8a03d94635a6b46cceff8dbfe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d958c8a03d94635a6b46cceff8dbfe8, State: Initialized, Role: FOLLOWER
I20260620 22:58:53.675957 4623 consensus_queue.cc:260] T 81d76a1d1f4842c685651cfb5661b741 P 6d958c8a03d94635a6b46cceff8dbfe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } }
I20260620 22:58:53.676501 4629 tablet_replica.cc:439] T 81d76a1d1f4842c685651cfb5661b741 P 6d958c8a03d94635a6b46cceff8dbfe8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } } }
I20260620 22:58:53.676914 4629 tablet_replica.cc:442] T 81d76a1d1f4842c685651cfb5661b741 P 6d958c8a03d94635a6b46cceff8dbfe8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 22:58:53.679813 4623 ts_tablet_manager.cc:1428] T 81d76a1d1f4842c685651cfb5661b741 P 6d958c8a03d94635a6b46cceff8dbfe8: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260620 22:58:53.682211 4625 tablet_bootstrap.cc:492] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed: No bootstrap required, opened a new log
I20260620 22:58:53.682435 4625 ts_tablet_manager.cc:1397] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed: Time spent bootstrapping tablet: real 0.028s user 0.006s sys 0.017s
I20260620 22:58:53.683256 4624 tablet_bootstrap.cc:492] T 81d76a1d1f4842c685651cfb5661b741 P 9d25123880f346369887bb834bc97880: No bootstrap required, opened a new log
I20260620 22:58:53.683185 4625 raft_consensus.cc:348] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } }
I20260620 22:58:53.683424 4624 ts_tablet_manager.cc:1397] T 81d76a1d1f4842c685651cfb5661b741 P 9d25123880f346369887bb834bc97880: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.004s
I20260620 22:58:53.683439 4625 raft_consensus.cc:374] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:58:53.683545 4625 raft_consensus.cc:729] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c6151ac8a714a2583c1310aa9a5fbed, State: Initialized, Role: FOLLOWER
I20260620 22:58:53.683840 4625 consensus_queue.cc:260] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } }
I20260620 22:58:53.684135 4624 raft_consensus.cc:348] T 81d76a1d1f4842c685651cfb5661b741 P 9d25123880f346369887bb834bc97880 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } }
I20260620 22:58:53.684365 4624 raft_consensus.cc:374] T 81d76a1d1f4842c685651cfb5661b741 P 9d25123880f346369887bb834bc97880 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:58:53.684429 4624 raft_consensus.cc:729] T 81d76a1d1f4842c685651cfb5661b741 P 9d25123880f346369887bb834bc97880 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d25123880f346369887bb834bc97880, State: Initialized, Role: FOLLOWER
I20260620 22:58:53.684319 4630 tablet_replica.cc:439] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } } }
I20260620 22:58:53.684530 4625 ts_tablet_manager.cc:1428] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260620 22:58:53.684662 4630 tablet_replica.cc:442] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 22:58:53.684657 4624 consensus_queue.cc:260] T 81d76a1d1f4842c685651cfb5661b741 P 9d25123880f346369887bb834bc97880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } }
I20260620 22:58:53.685166 4624 ts_tablet_manager.cc:1428] T 81d76a1d1f4842c685651cfb5661b741 P 9d25123880f346369887bb834bc97880: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260620 22:58:53.685127 4631 tablet_replica.cc:439] T 81d76a1d1f4842c685651cfb5661b741 P 9d25123880f346369887bb834bc97880: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } } }
I20260620 22:58:53.685470 4631 tablet_replica.cc:442] T 81d76a1d1f4842c685651cfb5661b741 P 9d25123880f346369887bb834bc97880: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 22:58:53.732263 4630 raft_consensus.cc:482] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 22:58:53.732501 4630 raft_consensus.cc:504] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } }
I20260620 22:58:53.733108 4630 leader_election.cc:288] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d958c8a03d94635a6b46cceff8dbfe8 (127.2.237.1:42623), 9d25123880f346369887bb834bc97880 (127.2.237.3:36923)
I20260620 22:58:53.733763 4554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81d76a1d1f4842c685651cfb5661b741" candidate_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d25123880f346369887bb834bc97880" is_pre_election: true
I20260620 22:58:53.733742 4275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81d76a1d1f4842c685651cfb5661b741" candidate_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" is_pre_election: true
I20260620 22:58:53.734033 4554 raft_consensus.cc:2454] T 81d76a1d1f4842c685651cfb5661b741 P 9d25123880f346369887bb834bc97880 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c6151ac8a714a2583c1310aa9a5fbed in term 0.
I20260620 22:58:53.734028 4275 raft_consensus.cc:2454] T 81d76a1d1f4842c685651cfb5661b741 P 6d958c8a03d94635a6b46cceff8dbfe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c6151ac8a714a2583c1310aa9a5fbed in term 0.
I20260620 22:58:53.734501 4346 leader_election.cc:302] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c6151ac8a714a2583c1310aa9a5fbed, 9d25123880f346369887bb834bc97880; no voters:
I20260620 22:58:53.734838 4630 raft_consensus.cc:2781] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 22:58:53.734951 4630 raft_consensus.cc:482] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 22:58:53.735491 4630 raft_consensus.cc:3037] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:58:53.738502 4630 raft_consensus.cc:504] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } }
I20260620 22:58:53.739156 4630 leader_election.cc:288] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed [CANDIDATE]: Term 1 election: Requested vote from peers 6d958c8a03d94635a6b46cceff8dbfe8 (127.2.237.1:42623), 9d25123880f346369887bb834bc97880 (127.2.237.3:36923)
I20260620 22:58:53.739629 4275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81d76a1d1f4842c685651cfb5661b741" candidate_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d958c8a03d94635a6b46cceff8dbfe8"
I20260620 22:58:53.739686 4554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81d76a1d1f4842c685651cfb5661b741" candidate_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d25123880f346369887bb834bc97880"
I20260620 22:58:53.739862 4275 raft_consensus.cc:3037] T 81d76a1d1f4842c685651cfb5661b741 P 6d958c8a03d94635a6b46cceff8dbfe8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:58:53.739935 4554 raft_consensus.cc:3037] T 81d76a1d1f4842c685651cfb5661b741 P 9d25123880f346369887bb834bc97880 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:58:53.742182 4554 raft_consensus.cc:2454] T 81d76a1d1f4842c685651cfb5661b741 P 9d25123880f346369887bb834bc97880 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c6151ac8a714a2583c1310aa9a5fbed in term 1.
I20260620 22:58:53.742705 4346 leader_election.cc:302] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c6151ac8a714a2583c1310aa9a5fbed, 9d25123880f346369887bb834bc97880; no voters:
I20260620 22:58:53.742796 4275 raft_consensus.cc:2454] T 81d76a1d1f4842c685651cfb5661b741 P 6d958c8a03d94635a6b46cceff8dbfe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c6151ac8a714a2583c1310aa9a5fbed in term 1.
I20260620 22:58:53.743134 4630 raft_consensus.cc:2781] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:58:53.743520 4630 raft_consensus.cc:686] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed [term 1 LEADER]: Becoming Leader. State: Replica: 3c6151ac8a714a2583c1310aa9a5fbed, State: Running, Role: LEADER
I20260620 22:58:53.743834 4630 consensus_queue.cc:237] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } }
I20260620 22:58:53.745994 4634 tablet_replica.cc:439] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed: TxnStatusTablet state changed. Reason: New leader 3c6151ac8a714a2583c1310aa9a5fbed. Latest consensus state: current_term: 1 leader_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } } }
I20260620 22:58:53.746333 4634 tablet_replica.cc:442] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed: This TxnStatusTablet replica's current role is: LEADER
I20260620 22:58:53.747035 4663 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260620 22:58:53.747295 4663 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260620 22:58:53.747812 4119 catalog_manager.cc:5495] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c6151ac8a714a2583c1310aa9a5fbed (127.2.237.2). New cstate: current_term: 1 leader_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } health_report { overall_health: UNKNOWN } } }
I20260620 22:58:53.776715 4620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:53.770783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55212 (local address 127.2.237.2:39403)
0620 22:58:53.770928 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:53.770932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:53.770970 (+ 38us) server_negotiation.cc:407] Connection header received
0620 22:58:53.771302 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:53.771306 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:53.771373 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:53.771448 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 22:58:53.771868 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.772811 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.774671 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.775465 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.775560 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:53.776077 (+ 517us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 22:58:53.776189 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 22:58:53.776250 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:53.776438 (+ 188us) server_negotiation.cc:299] Negotiation successful
0620 22:58:53.776496 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260620 22:58:53.776885 4658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:53.770639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:39403 (local address 127.0.0.1:55212)
0620 22:58:53.770889 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:53.770912 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:53.771078 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:53.771507 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:53.771513 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:53.771527 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 22:58:53.771727 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:53.771738 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.772942 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.772948 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:53.774513 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:53.774529 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.775625 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.775632 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:53.775845 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:53.775919 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 22:58:53.776286 (+ 367us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 22:58:53.776319 (+ 33us) client_negotiation.cc:769] Sending connection context
0620 22:58:53.776417 (+ 98us) client_negotiation.cc:240] Negotiation successful
0620 22:58:53.776583 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260620 22:58:53.780691 4554 raft_consensus.cc:1261] T 81d76a1d1f4842c685651cfb5661b741 P 9d25123880f346369887bb834bc97880 [term 1 FOLLOWER]: Refusing update from remote peer 3c6151ac8a714a2583c1310aa9a5fbed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:58:53.780751 4275 raft_consensus.cc:1261] T 81d76a1d1f4842c685651cfb5661b741 P 6d958c8a03d94635a6b46cceff8dbfe8 [term 1 FOLLOWER]: Refusing update from remote peer 3c6151ac8a714a2583c1310aa9a5fbed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:58:53.781284 4630 consensus_queue.cc:1035] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:58:53.781620 4634 consensus_queue.cc:1035] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:58:53.785676 4629 tablet_replica.cc:439] T 81d76a1d1f4842c685651cfb5661b741 P 6d958c8a03d94635a6b46cceff8dbfe8: TxnStatusTablet state changed. Reason: New leader 3c6151ac8a714a2583c1310aa9a5fbed. Latest consensus state: current_term: 1 leader_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } } }
I20260620 22:58:53.786302 4629 tablet_replica.cc:442] T 81d76a1d1f4842c685651cfb5661b741 P 6d958c8a03d94635a6b46cceff8dbfe8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 22:58:53.787344 4631 tablet_replica.cc:439] T 81d76a1d1f4842c685651cfb5661b741 P 9d25123880f346369887bb834bc97880: TxnStatusTablet state changed. Reason: New leader 3c6151ac8a714a2583c1310aa9a5fbed. Latest consensus state: current_term: 1 leader_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } } }
I20260620 22:58:53.787595 4631 tablet_replica.cc:442] T 81d76a1d1f4842c685651cfb5661b741 P 9d25123880f346369887bb834bc97880: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 22:58:53.789337 4630 tablet_replica.cc:439] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } } }
I20260620 22:58:53.789623 4630 tablet_replica.cc:442] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed: This TxnStatusTablet replica's current role is: LEADER
I20260620 22:58:53.789821 4637 mvcc.cc:205] Tried to move back new op lower bound from 7299056983159193600 to 7299056983016902656. Current Snapshot: MvccSnapshot[applied={T|T < 7299056983159193600 or (T in {7299056983159193600})}]
I20260620 22:58:53.791580 4634 tablet_replica.cc:439] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } } }
I20260620 22:58:53.791779 4634 tablet_replica.cc:442] T 81d76a1d1f4842c685651cfb5661b741 P 3c6151ac8a714a2583c1310aa9a5fbed: This TxnStatusTablet replica's current role is: LEADER
I20260620 22:58:53.792755 4631 tablet_replica.cc:439] T 81d76a1d1f4842c685651cfb5661b741 P 9d25123880f346369887bb834bc97880: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } } }
I20260620 22:58:53.792986 4631 tablet_replica.cc:442] T 81d76a1d1f4842c685651cfb5661b741 P 9d25123880f346369887bb834bc97880: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 22:58:53.798411 4629 tablet_replica.cc:439] T 81d76a1d1f4842c685651cfb5661b741 P 6d958c8a03d94635a6b46cceff8dbfe8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 } } peers { permanent_uuid: "3c6151ac8a714a2583c1310aa9a5fbed" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 39403 } } peers { permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 } } }
I20260620 22:58:53.798658 4629 tablet_replica.cc:442] T 81d76a1d1f4842c685651cfb5661b741 P 6d958c8a03d94635a6b46cceff8dbfe8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 22:58:53.812932 4620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:53.807063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:43511 (local address 127.2.237.2:39403)
0620 22:58:53.807299 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:53.807307 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:53.807565 (+ 258us) server_negotiation.cc:407] Connection header received
0620 22:58:53.807774 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:53.807778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:53.807843 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:53.807928 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 22:58:53.808861 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.809604 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.811121 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.811555 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.811715 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:53.812177 (+ 462us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 22:58:53.812291 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 22:58:53.812391 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:53.812543 (+ 152us) server_negotiation.cc:299] Negotiation successful
0620 22:58:53.812609 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":64}
I20260620 22:58:53.813079 4670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:53.806785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:39403 (local address 127.2.237.2:43511)
0620 22:58:53.807471 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:53.807489 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:53.807646 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:53.807948 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:53.807952 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:53.807962 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 22:58:53.808695 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:53.808708 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.809745 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.809753 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:53.810981 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:53.810992 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.811737 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.811743 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:53.811952 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:53.812000 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 22:58:53.812388 (+ 388us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 22:58:53.812407 (+ 19us) client_negotiation.cc:769] Sending connection context
0620 22:58:53.812519 (+ 112us) client_negotiation.cc:240] Negotiation successful
0620 22:58:53.812646 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"mutex_wait_us":18,"thread_start_us":360,"threads_started":1}
I20260620 22:58:53.823007 4275 raft_consensus.cc:1261] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 6d958c8a03d94635a6b46cceff8dbfe8 [term 1 FOLLOWER]: Refusing update from remote peer 3c6151ac8a714a2583c1310aa9a5fbed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:58:53.823118 4554 raft_consensus.cc:1261] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 9d25123880f346369887bb834bc97880 [term 1 FOLLOWER]: Refusing update from remote peer 3c6151ac8a714a2583c1310aa9a5fbed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:58:53.823663 4630 consensus_queue.cc:1035] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d25123880f346369887bb834bc97880" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:58:53.823978 4667 consensus_queue.cc:1035] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d958c8a03d94635a6b46cceff8dbfe8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:58:53.841167 4639 mvcc.cc:205] Tried to move back new op lower bound from 7299056983334731776 to 7299056982339407872. Current Snapshot: MvccSnapshot[applied={T|T < 7299056983334731776}]
I20260620 22:58:53.880220 4118 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 22:58:53.880520 4118 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 22:58:53.883683 4118 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2e2c41911b804b5ea76ef0485b087abd: Sending DeleteTablet for 3 replicas of tablet 937a808ac7ee43ffbaf2be5dd1a34ac7
I20260620 22:58:53.884490 4256 tablet_service.cc:1514] Processing DeleteTablet for tablet 937a808ac7ee43ffbaf2be5dd1a34ac7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:58:53 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:58328
I20260620 22:58:53.884505 4395 tablet_service.cc:1514] Processing DeleteTablet for tablet 937a808ac7ee43ffbaf2be5dd1a34ac7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:58:53 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:55196
I20260620 22:58:53.884756 4646 tablet_replica.cc:330] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed: stopping tablet replica
I20260620 22:58:53.884748 4648 tablet_replica.cc:330] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 6d958c8a03d94635a6b46cceff8dbfe8: stopping tablet replica
I20260620 22:58:53.884934 4646 raft_consensus.cc:2229] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed [term 1 LEADER]: Raft consensus shutting down.
I20260620 22:58:53.884963 4648 raft_consensus.cc:2229] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 6d958c8a03d94635a6b46cceff8dbfe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 22:58:53.885134 4648 raft_consensus.cc:2258] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 6d958c8a03d94635a6b46cceff8dbfe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:58:53.885262 4646 raft_consensus.cc:2258] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:58:53.885386 4534 tablet_service.cc:1514] Processing DeleteTablet for tablet 937a808ac7ee43ffbaf2be5dd1a34ac7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:58:53 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:53474
I20260620 22:58:53.885712 4647 tablet_replica.cc:330] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 9d25123880f346369887bb834bc97880: stopping tablet replica
I20260620 22:58:53.885929 4647 raft_consensus.cc:2229] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 9d25123880f346369887bb834bc97880 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 22:58:53.886149 4647 raft_consensus.cc:2258] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 9d25123880f346369887bb834bc97880 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:58:53.886420 4648 ts_tablet_manager.cc:1905] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 6d958c8a03d94635a6b46cceff8dbfe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:58:53.886869 4646 ts_tablet_manager.cc:1905] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:58:53.887529 4647 ts_tablet_manager.cc:1905] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 9d25123880f346369887bb834bc97880: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:58:53.893574 4648 ts_tablet_manager.cc:1918] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 6d958c8a03d94635a6b46cceff8dbfe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 22:58:53.893731 4648 log.cc:1196] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 6d958c8a03d94635a6b46cceff8dbfe8: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-0/wal/wals/937a808ac7ee43ffbaf2be5dd1a34ac7
I20260620 22:58:53.893778 4646 ts_tablet_manager.cc:1918] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 22:58:53.894070 4646 log.cc:1196] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-1/wal/wals/937a808ac7ee43ffbaf2be5dd1a34ac7
I20260620 22:58:53.894163 4648 ts_tablet_manager.cc:1939] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 6d958c8a03d94635a6b46cceff8dbfe8: Deleting consensus metadata
I20260620 22:58:53.894645 4646 ts_tablet_manager.cc:1939] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 3c6151ac8a714a2583c1310aa9a5fbed: Deleting consensus metadata
I20260620 22:58:53.895321 4103 catalog_manager.cc:4841] TS 6d958c8a03d94635a6b46cceff8dbfe8 (127.2.237.1:42623): tablet 937a808ac7ee43ffbaf2be5dd1a34ac7 (table test-table [id=44410520bc044d1589ff11b81e64e305]) successfully deleted
I20260620 22:58:53.895743 4105 catalog_manager.cc:4841] TS 3c6151ac8a714a2583c1310aa9a5fbed (127.2.237.2:39403): tablet 937a808ac7ee43ffbaf2be5dd1a34ac7 (table test-table [id=44410520bc044d1589ff11b81e64e305]) successfully deleted
I20260620 22:58:53.896847 4647 ts_tablet_manager.cc:1918] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 9d25123880f346369887bb834bc97880: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 22:58:53.897501 4647 log.cc:1196] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 9d25123880f346369887bb834bc97880: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781996318807807-2996-0/minicluster-data/ts-2/wal/wals/937a808ac7ee43ffbaf2be5dd1a34ac7
I20260620 22:58:53.897987 4647 ts_tablet_manager.cc:1939] T 937a808ac7ee43ffbaf2be5dd1a34ac7 P 9d25123880f346369887bb834bc97880: Deleting consensus metadata
I20260620 22:58:53.899137 4103 catalog_manager.cc:4841] TS 9d25123880f346369887bb834bc97880 (127.2.237.3:36923): tablet 937a808ac7ee43ffbaf2be5dd1a34ac7 (table test-table [id=44410520bc044d1589ff11b81e64e305]) successfully deleted
Jun 20 22:58:53 dist-test-slave-f63b krb5kdc[4074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996333, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.237.1@KRBTEST.COM
I20260620 22:58:53.917490 4677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:53.897978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58342 (local address 127.2.237.1:42623)
0620 22:58:53.899551 (+ 1573us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:53.899561 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:53.900178 (+ 617us) server_negotiation.cc:407] Connection header received
0620 22:58:53.900374 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:53.900381 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:53.900461 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:53.900587 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:53.901862 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.902978 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.904164 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.905191 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.905325 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:53.910342 (+ 5017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:53.910360 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:53.910383 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:53.910430 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:53.914350 (+ 3920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:53.915110 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:53.915120 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:53.915124 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:53.915310 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:53.915922 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:53.915928 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:53.915932 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:53.916384 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:53.916698 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:53.916864 (+ 166us) server_negotiation.cc:299] Negotiation successful
0620 22:58:53.917093 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1405,"thread_start_us":1367,"threads_started":1}
W20260620 22:58:53.918282 4318 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58342
I20260620 22:58:53.934880 4214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:53.921250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43470 (local address 127.2.237.62:39831)
0620 22:58:53.921527 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:53.921536 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:53.921920 (+ 384us) server_negotiation.cc:407] Connection header received
0620 22:58:53.922044 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:53.922050 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:53.922133 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:53.922240 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:53.923432 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.924604 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.925841 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.926963 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.927094 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:53.929311 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:53.929318 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:53.929323 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:53.929362 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:53.932477 (+ 3115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:53.933038 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:53.933043 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:53.933047 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:53.933198 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:53.933519 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:53.933524 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:53.933527 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:53.933845 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:53.934102 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:53.934281 (+ 179us) server_negotiation.cc:299] Negotiation successful
0620 22:58:53.934486 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260620 22:58:53.935451 4118 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43470
I20260620 22:58:53.938073 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 4190
I20260620 22:58:53.966010 4677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:53.939013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58344 (local address 127.2.237.1:42623)
0620 22:58:53.939226 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:53.939232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:53.953881 (+ 14649us) server_negotiation.cc:407] Connection header received
0620 22:58:53.954083 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:53.954087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:53.954156 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:53.954273 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:53.955554 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.956251 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.957939 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.958367 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.958506 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:53.960812 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:53.960817 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:53.960821 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:53.960853 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:53.963505 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:53.964173 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:53.964183 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:53.964185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:53.964307 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:53.964678 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:53.964683 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:53.964687 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:53.964942 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:53.965197 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:53.965476 (+ 279us) server_negotiation.cc:299] Negotiation successful
0620 22:58:53.965691 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260620 22:58:53.966711 4318 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58344
W20260620 22:58:53.967319 2996 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 4190: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260620 22:58:53.967617 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 4190
I20260620 22:58:53.986837 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 4330
Jun 20 22:58:53 dist-test-slave-f63b krb5kdc[4074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996333, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.237.2@KRBTEST.COM
I20260620 22:58:54.003885 4620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:53.987615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55220 (local address 127.2.237.2:39403)
0620 22:58:53.987773 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:53.987778 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:53.987853 (+ 75us) server_negotiation.cc:407] Connection header received
0620 22:58:53.988052 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:53.988056 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:53.988123 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:53.988204 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:53.989747 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.990549 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.992447 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:53.992964 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:53.993056 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:53.997834 (+ 4778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:53.997858 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:53.997880 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:53.997920 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:54.000954 (+ 3034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:54.001598 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:54.001605 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:54.001607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:54.001734 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:54.002154 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:54.002159 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:54.002162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:54.002666 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:54.002987 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:54.003293 (+ 306us) server_negotiation.cc:299] Negotiation successful
0620 22:58:54.003530 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260620 22:58:54.004597 4460 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55220
W20260620 22:58:54.005275 2996 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 4330: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260620 22:58:54.005455 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 4330
I20260620 22:58:54.027026 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 4469
Jun 20 22:58:54 dist-test-slave-f63b krb5kdc[4074](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996333, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.237.3@KRBTEST.COM
I20260620 22:58:54.041935 4682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:54.027724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53476 (local address 127.2.237.3:36923)
0620 22:58:54.028260 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:54.028265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:54.028282 (+ 17us) server_negotiation.cc:407] Connection header received
0620 22:58:54.028334 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:54.028338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:54.028394 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:54.028508 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:54.029675 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:54.030417 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:54.031635 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:54.032078 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:54.032187 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:54.036243 (+ 4056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:54.036258 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:54.036278 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:54.036313 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:54.039310 (+ 2997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:54.039985 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:54.039993 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:54.039995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:54.040129 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:54.040466 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:54.040470 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:54.040472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:54.040867 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:54.041108 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:54.041308 (+ 200us) server_negotiation.cc:299] Negotiation successful
0620 22:58:54.041489 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":330,"threads_started":1}
W20260620 22:58:54.042699 4599 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53476
W20260620 22:58:54.043341 2996 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 4469: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260620 22:58:54.043475 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 4469
I20260620 22:58:54.059940 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 4087
I20260620 22:58:54.061059 4159 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:58:54.175380 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 4087
2026-06-20T22:58:54Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (7107 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 22:58:54 dist-test-slave-f63b krb5kdc[4686](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 22:58:54 dist-test-slave-f63b krb5kdc[4686](info): set up 2 sockets
Jun 20 22:58:54 dist-test-slave-f63b krb5kdc[4686](info): commencing operation
krb5kdc: starting...
W20260620 22:58:55.255220 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.000s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 22:58:55 dist-test-slave-f63b krb5kdc[4686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T22:58:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T22:58:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
I20260620 22:58:55.437942 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:37999
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:41607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:37999
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:58:55.746182 4699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:58:55.746434 4699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:58:55.746587 4699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:58:55.753649 4699 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 22:58:55.753744 4699 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260620 22:58:55.753837 4699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:58:55.753958 4699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:58:55.754040 4699 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 22:58:55.754149 4699 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 22:58:55.763696 4699 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:41607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:37999
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:37999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:58:55.764925 4699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:58:55.766251 4699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:58:55.777729 4704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:55.777695 4707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:55.777653 4705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:58:55.778527 4699 server_base.cc:1034] running on GCE node
I20260620 22:58:55.779246 4699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:58:55.780498 4699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:58:55.781710 4699 hybrid_clock.cc:648] HybridClock initialized: now 1781996335781660 us; error 24 us; skew 500 ppm
I20260620 22:58:55.782049 4699 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:58:55 dist-test-slave-f63b krb5kdc[4686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996335, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:58:55.788499 4699 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 22:58:55.790033 4699 webserver.cc:460] Webserver started at http://127.2.237.62:44233/ using document root <none> and password file <none>
I20260620 22:58:55.790835 4699 fs_manager.cc:359] Metadata directory not provided
I20260620 22:58:55.790957 4699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:58:55.791294 4699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:58:55.794323 4699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "8ba9bd49121f41eb97acc05b4e83ac70"
format_stamp: "Formatted at 2026-06-20 22:58:55 on dist-test-slave-f63b"
server_key: "7d5ec4cb4db8ed979802449403e96a7d"
server_key_iv: "d6d46483c6aaf55b5366cb8eed052c10"
server_key_version: "encryptionkey@0"
I20260620 22:58:55.795168 4699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "8ba9bd49121f41eb97acc05b4e83ac70"
format_stamp: "Formatted at 2026-06-20 22:58:55 on dist-test-slave-f63b"
server_key: "7d5ec4cb4db8ed979802449403e96a7d"
server_key_iv: "d6d46483c6aaf55b5366cb8eed052c10"
server_key_version: "encryptionkey@0"
I20260620 22:58:55.800192 4699 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260620 22:58:55.803382 4714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:55.804663 4699 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260620 22:58:55.804844 4699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "8ba9bd49121f41eb97acc05b4e83ac70"
format_stamp: "Formatted at 2026-06-20 22:58:55 on dist-test-slave-f63b"
server_key: "7d5ec4cb4db8ed979802449403e96a7d"
server_key_iv: "d6d46483c6aaf55b5366cb8eed052c10"
server_key_version: "encryptionkey@0"
I20260620 22:58:55.805048 4699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:58:55.825501 4699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:58:55.829366 4699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:58:55.829635 4699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:58:55.850256 4699 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:37999
I20260620 22:58:55.850265 4775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:37999 every 8 connection(s)
I20260620 22:58:55.851606 4699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 22:58:55.855155 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 4699
I20260620 22:58:55.855372 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 22:58:55.855424 4776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:58:55.856009 2996 external_mini_cluster.cc:1428] Setting key 5774eee16792c7bdb2286ebe29c34057
I20260620 22:58:55.869135 4776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ba9bd49121f41eb97acc05b4e83ac70: Bootstrap starting.
Jun 20 22:58:55 dist-test-slave-f63b krb5kdc[4686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:58:55.872471 4776 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ba9bd49121f41eb97acc05b4e83ac70: Neither blocks nor log segments found. Creating new log.
I20260620 22:58:55.873807 4776 log.cc:824] T 00000000000000000000000000000000 P 8ba9bd49121f41eb97acc05b4e83ac70: Log is configured to *not* fsync() on all Append() calls
I20260620 22:58:55.876931 4776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ba9bd49121f41eb97acc05b4e83ac70: No bootstrap required, opened a new log
I20260620 22:58:55.879626 4779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:55.857962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37896 (local address 127.2.237.62:37999)
0620 22:58:55.858788 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:55.858803 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:55.858850 (+ 47us) server_negotiation.cc:407] Connection header received
0620 22:58:55.860448 (+ 1598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:55.860475 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:55.860983 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:55.861434 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:55.863038 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:55.864886 (+ 1848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:55.866181 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:55.867304 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:55.867502 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:55.871470 (+ 3968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:55.871482 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:55.871502 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:55.871556 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:55.875300 (+ 3744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:55.875997 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:55.876005 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:55.876012 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:55.876227 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:55.876600 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:55.876605 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:55.876608 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:55.877238 (+ 630us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:55.877544 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:55.877903 (+ 359us) server_negotiation.cc:299] Negotiation successful
0620 22:58:55.878312 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":357,"threads_started":1}
I20260620 22:58:55.885624 4776 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ba9bd49121f41eb97acc05b4e83ac70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba9bd49121f41eb97acc05b4e83ac70" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37999 } }
I20260620 22:58:55.885946 4776 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ba9bd49121f41eb97acc05b4e83ac70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:58:55.886022 4776 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ba9bd49121f41eb97acc05b4e83ac70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ba9bd49121f41eb97acc05b4e83ac70, State: Initialized, Role: FOLLOWER
I20260620 22:58:55.886490 4776 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ba9bd49121f41eb97acc05b4e83ac70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba9bd49121f41eb97acc05b4e83ac70" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37999 } }
I20260620 22:58:55.886777 4776 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ba9bd49121f41eb97acc05b4e83ac70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 22:58:55.886911 4776 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ba9bd49121f41eb97acc05b4e83ac70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 22:58:55.887084 4776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ba9bd49121f41eb97acc05b4e83ac70 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:58:55.889539 4776 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ba9bd49121f41eb97acc05b4e83ac70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba9bd49121f41eb97acc05b4e83ac70" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37999 } }
I20260620 22:58:55.890022 4776 leader_election.cc:302] T 00000000000000000000000000000000 P 8ba9bd49121f41eb97acc05b4e83ac70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ba9bd49121f41eb97acc05b4e83ac70; no voters:
I20260620 22:58:55.890559 4776 leader_election.cc:288] T 00000000000000000000000000000000 P 8ba9bd49121f41eb97acc05b4e83ac70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 22:58:55.890753 4781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ba9bd49121f41eb97acc05b4e83ac70 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:58:55.891433 4781 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ba9bd49121f41eb97acc05b4e83ac70 [term 1 LEADER]: Becoming Leader. State: Replica: 8ba9bd49121f41eb97acc05b4e83ac70, State: Running, Role: LEADER
I20260620 22:58:55.891988 4781 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ba9bd49121f41eb97acc05b4e83ac70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba9bd49121f41eb97acc05b4e83ac70" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37999 } }
I20260620 22:58:55.892802 4776 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ba9bd49121f41eb97acc05b4e83ac70 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 22:58:55.894724 4783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ba9bd49121f41eb97acc05b4e83ac70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ba9bd49121f41eb97acc05b4e83ac70. Latest consensus state: current_term: 1 leader_uuid: "8ba9bd49121f41eb97acc05b4e83ac70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba9bd49121f41eb97acc05b4e83ac70" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37999 } } }
I20260620 22:58:55.894786 4782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ba9bd49121f41eb97acc05b4e83ac70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ba9bd49121f41eb97acc05b4e83ac70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba9bd49121f41eb97acc05b4e83ac70" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37999 } } }
I20260620 22:58:55.895260 4782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ba9bd49121f41eb97acc05b4e83ac70 [sys.catalog]: This master's current role is: LEADER
I20260620 22:58:55.896274 4783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ba9bd49121f41eb97acc05b4e83ac70 [sys.catalog]: This master's current role is: LEADER
I20260620 22:58:55.898149 4788 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 22:58:55.903162 4788 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 22:58:55.911507 4788 catalog_manager.cc:1347] Generated new cluster ID: 51bb0495958546869d4b365871ca1512
I20260620 22:58:55.911609 4788 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 22:58:55.926044 4788 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 22:58:55.927446 4788 catalog_manager.cc:1504] Loading token signing keys...
I20260620 22:58:55.932224 4788 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ba9bd49121f41eb97acc05b4e83ac70: Generated new TSK 0
I20260620 22:58:55.933034 4788 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.237.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
I20260620 22:58:55.984017 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:0
--local_ip_for_outbound_sockets=127.2.237.1
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:37999
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:41607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:58:56.328079 4802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:58:56.328370 4802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:58:56.328491 4802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:58:56.335736 4802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:58:56.335855 4802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:58:56.335943 4802 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260620 22:58:56.335996 4802 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 22:58:56.336063 4802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 22:58:56.345872 4802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:41607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.237.62:37999
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:58:56.347184 4802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:58:56.348489 4802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:58:56.361582 4808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:56.361699 4807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:56.361660 4810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:58:56.362984 4802 server_base.cc:1034] running on GCE node
I20260620 22:58:56.363507 4802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:58:56.364348 4802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:58:56.365530 4802 hybrid_clock.cc:648] HybridClock initialized: now 1781996336365492 us; error 34 us; skew 500 ppm
I20260620 22:58:56.365902 4802 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:58:56 dist-test-slave-f63b krb5kdc[4686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996336, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:58:56.369063 4802 init.cc:377] Logged in from keytab as kudu/127.2.237.1@KRBTEST.COM (short username kudu)
I20260620 22:58:56.370489 4802 webserver.cc:460] Webserver started at http://127.2.237.1:42257/ using document root <none> and password file <none>
I20260620 22:58:56.371202 4802 fs_manager.cc:359] Metadata directory not provided
I20260620 22:58:56.371352 4802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:58:56.371660 4802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:58:56.374523 4802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-0/data/instance:
uuid: "f832345cd08e4521bff81fe662f9e938"
format_stamp: "Formatted at 2026-06-20 22:58:56 on dist-test-slave-f63b"
server_key: "c044e7f6d96e09bf321c6d358d27cabf"
server_key_iv: "52095498461f6d5c91fa98b1ae7c0ee8"
server_key_version: "encryptionkey@0"
I20260620 22:58:56.375365 4802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance:
uuid: "f832345cd08e4521bff81fe662f9e938"
format_stamp: "Formatted at 2026-06-20 22:58:56 on dist-test-slave-f63b"
server_key: "c044e7f6d96e09bf321c6d358d27cabf"
server_key_iv: "52095498461f6d5c91fa98b1ae7c0ee8"
server_key_version: "encryptionkey@0"
I20260620 22:58:56.380960 4802 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260620 22:58:56.384377 4817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:56.385741 4802 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 22:58:56.385946 4802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "f832345cd08e4521bff81fe662f9e938"
format_stamp: "Formatted at 2026-06-20 22:58:56 on dist-test-slave-f63b"
server_key: "c044e7f6d96e09bf321c6d358d27cabf"
server_key_iv: "52095498461f6d5c91fa98b1ae7c0ee8"
server_key_version: "encryptionkey@0"
I20260620 22:58:56.386137 4802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:58:56.416575 4802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:58:56.420524 4802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:58:56.420831 4802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:58:56.423511 4802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:58:56.423666 4802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:56.423807 4802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:58:56.423911 4802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 22:58:56 dist-test-slave-f63b krb5kdc[4686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996336, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:58:56.447155 4826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:56.424060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:52871 (local address 127.2.237.62:37999)
0620 22:58:56.424783 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:56.424790 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:56.425982 (+ 1192us) server_negotiation.cc:407] Connection header received
0620 22:58:56.429308 (+ 3326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:56.429315 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:56.429413 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:56.429548 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:56.431779 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:56.433127 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:56.435186 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:56.435630 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:56.435765 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:56.439966 (+ 4201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:56.439974 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:56.439979 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:56.440021 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:56.443517 (+ 3496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:56.444637 (+ 1120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:56.444642 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:56.444646 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:56.444766 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:56.445236 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:56.445241 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:56.445245 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:56.445509 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:56.445662 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:56.446540 (+ 878us) server_negotiation.cc:299] Negotiation successful
0620 22:58:56.446787 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":478,"threads_started":1}
I20260620 22:58:56.448745 4827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:56.424787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37999 (local address 127.2.237.1:52871)
0620 22:58:56.425748 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:56.425808 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:56.428942 (+ 3134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:56.429829 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:56.429843 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:56.430671 (+ 828us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:58:56.431568 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:56.431604 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:56.433302 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:56.433306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:56.434970 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:56.434996 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:56.435850 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:56.435858 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:56.436066 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:56.436716 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:58:56.436737 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:58:56.439694 (+ 2957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:58:56.443665 (+ 3971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:56.443678 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:56.443697 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:56.444459 (+ 762us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:56.444880 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:56.444885 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:56.444892 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:56.445100 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:56.445634 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:58:56.445642 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:58:56.446037 (+ 395us) client_negotiation.cc:769] Sending connection context
0620 22:58:56.446428 (+ 391us) client_negotiation.cc:240] Negotiation successful
0620 22:58:56.447014 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":404,"threads_started":1}
I20260620 22:58:56.470813 4826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:56.456242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:50129 (local address 127.2.237.62:37999)
0620 22:58:56.456426 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:56.456432 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:56.456587 (+ 155us) server_negotiation.cc:407] Connection header received
0620 22:58:56.456765 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:56.456773 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:56.456877 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:56.457009 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:56.457833 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:56.459201 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:56.461113 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:56.461495 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:56.461655 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:56.463849 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:56.463858 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:56.463866 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:56.463907 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:56.467805 (+ 3898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:56.468557 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:56.468562 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:56.468565 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:56.468730 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:56.469089 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:56.469097 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:56.469103 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:56.469527 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:56.469709 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:56.470207 (+ 498us) server_negotiation.cc:299] Negotiation successful
0620 22:58:56.470432 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 22:58:56.470963 4893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:56.455780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37999 (local address 127.2.237.1:50129)
0620 22:58:56.456460 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:56.456484 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:56.456623 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:56.457023 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:56.457026 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:56.457426 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:58:56.457696 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:56.457704 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:56.459428 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:56.459434 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:56.460941 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:56.460955 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:56.461695 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:56.461701 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:56.461851 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:56.462466 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:58:56.462482 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:58:56.463544 (+ 1062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:58:56.467985 (+ 4441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:56.467991 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:56.467997 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:56.468422 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:56.468843 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:56.468846 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:56.468849 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:56.468970 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:56.469652 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:58:56.469658 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:58:56.469932 (+ 274us) client_negotiation.cc:769] Sending connection context
0620 22:58:56.470101 (+ 169us) client_negotiation.cc:240] Negotiation successful
0620 22:58:56.470453 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":652,"threads_started":1}
I20260620 22:58:56.478199 4802 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:33195
I20260620 22:58:56.478229 4937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:33195 every 8 connection(s)
I20260620 22:58:56.479493 4802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 22:58:56.481498 4938 heartbeater.cc:347] Connected to a master server at 127.2.237.62:37999
I20260620 22:58:56.481722 4938 heartbeater.cc:464] Registering TS with master...
I20260620 22:58:56.482358 4938 heartbeater.cc:511] Master 127.2.237.62:37999 requested a full tablet report, sending...
I20260620 22:58:56.484244 4731 ts_manager.cc:194] Registered new tserver with Master: f832345cd08e4521bff81fe662f9e938 (127.2.237.1:33195)
I20260620 22:58:56.485114 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 4802
I20260620 22:58:56.485308 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance
I20260620 22:58:56.486052 2996 external_mini_cluster.cc:1428] Setting key ea6ecddcf34423951836471fa70de095
I20260620 22:58:56.486331 4731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.1@KRBTEST.COM'} at 127.2.237.1:52871
WARNING: no policy specified for kudu/127.2.237.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
I20260620 22:58:56.523940 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:0
--local_ip_for_outbound_sockets=127.2.237.2
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:37999
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:41607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:58:56.844645 4942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:58:56.844925 4942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:58:56.845028 4942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:58:56.852135 4942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:58:56.852244 4942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:58:56.852427 4942 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260620 22:58:56.852506 4942 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 22:58:56.852638 4942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 22:58:56.862272 4942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:41607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.237.62:37999
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:58:56.863540 4942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:58:56.864806 4942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:58:56.877614 4948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:56.877595 4947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:56.877635 4950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:58:56.878263 4942 server_base.cc:1034] running on GCE node
I20260620 22:58:56.878916 4942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:58:56.879813 4942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:58:56.881042 4942 hybrid_clock.cc:648] HybridClock initialized: now 1781996336881008 us; error 51 us; skew 500 ppm
I20260620 22:58:56.881348 4942 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:58:56 dist-test-slave-f63b krb5kdc[4686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996336, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:58:56.884519 4942 init.cc:377] Logged in from keytab as kudu/127.2.237.2@KRBTEST.COM (short username kudu)
I20260620 22:58:56.885878 4942 webserver.cc:460] Webserver started at http://127.2.237.2:37695/ using document root <none> and password file <none>
I20260620 22:58:56.886696 4942 fs_manager.cc:359] Metadata directory not provided
I20260620 22:58:56.886838 4942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:58:56.887156 4942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:58:56.890079 4942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-1/data/instance:
uuid: "a0683d16ce0b41fab23ab9d548a9509b"
format_stamp: "Formatted at 2026-06-20 22:58:56 on dist-test-slave-f63b"
server_key: "7d5becf500d86fe052226149c8c8c34f"
server_key_iv: "d645af291b9be8f12c2ba4d1ce3ae3f7"
server_key_version: "encryptionkey@0"
I20260620 22:58:56.890889 4942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance:
uuid: "a0683d16ce0b41fab23ab9d548a9509b"
format_stamp: "Formatted at 2026-06-20 22:58:56 on dist-test-slave-f63b"
server_key: "7d5becf500d86fe052226149c8c8c34f"
server_key_iv: "d645af291b9be8f12c2ba4d1ce3ae3f7"
server_key_version: "encryptionkey@0"
I20260620 22:58:56.896173 4942 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260620 22:58:56.899609 4957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:56.900945 4942 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 22:58:56.901109 4942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "a0683d16ce0b41fab23ab9d548a9509b"
format_stamp: "Formatted at 2026-06-20 22:58:56 on dist-test-slave-f63b"
server_key: "7d5becf500d86fe052226149c8c8c34f"
server_key_iv: "d645af291b9be8f12c2ba4d1ce3ae3f7"
server_key_version: "encryptionkey@0"
I20260620 22:58:56.901307 4942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:58:56.921453 4942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:58:56.925704 4942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:58:56.925992 4942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:58:56.928557 4942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:58:56.928704 4942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:56.928835 4942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:58:56.928936 4942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 22:58:56 dist-test-slave-f63b krb5kdc[4686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996336, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:58:56.952323 4826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:56.929287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:38441 (local address 127.2.237.62:37999)
0620 22:58:56.929523 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:56.929533 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:56.930929 (+ 1396us) server_negotiation.cc:407] Connection header received
0620 22:58:56.933159 (+ 2230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:56.933166 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:56.933245 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:56.933361 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:56.935633 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:56.936917 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:56.939443 (+ 2526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:56.940068 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:56.940229 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:56.944955 (+ 4726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:56.944964 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:56.944969 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:56.945015 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:56.948544 (+ 3529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:56.949682 (+ 1138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:56.949689 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:56.949693 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:56.949822 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:56.950436 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:56.950444 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:56.950449 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:56.950821 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:56.951009 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:56.951735 (+ 726us) server_negotiation.cc:299] Negotiation successful
0620 22:58:56.951969 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 22:58:56.953465 4966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:56.929822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37999 (local address 127.2.237.2:38441)
0620 22:58:56.930747 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:56.930792 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:56.932830 (+ 2038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:56.933638 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:56.933656 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:56.934484 (+ 828us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:58:56.935418 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:56.935454 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:56.937107 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:56.937115 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:56.939289 (+ 2174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:56.939301 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:56.940556 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:56.940574 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:56.940749 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:56.941583 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:58:56.941604 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:58:56.944682 (+ 3078us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:58:56.948691 (+ 4009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:56.948704 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:56.948729 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:56.949521 (+ 792us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:56.949944 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:56.949953 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:56.949959 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:56.950280 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:56.950976 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:58:56.950985 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:58:56.951358 (+ 373us) client_negotiation.cc:769] Sending connection context
0620 22:58:56.951631 (+ 273us) client_negotiation.cc:240] Negotiation successful
0620 22:58:56.952203 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":416,"threads_started":1}
I20260620 22:58:56.975728 4826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:56.959168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:54723 (local address 127.2.237.62:37999)
0620 22:58:56.959344 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:56.959348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:56.959974 (+ 626us) server_negotiation.cc:407] Connection header received
0620 22:58:56.960221 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:56.960227 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:56.960300 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:56.960402 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:56.961894 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:56.963039 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:56.965178 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:56.965810 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:56.965950 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:56.968218 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:56.968228 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:56.968235 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:56.968284 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:56.972473 (+ 4189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:56.973342 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:56.973347 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:56.973352 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:56.973545 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:56.973996 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:56.974003 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:56.974009 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:56.974549 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:56.974735 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:56.975279 (+ 544us) server_negotiation.cc:299] Negotiation successful
0620 22:58:56.975509 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 22:58:56.976282 5032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:56.959220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37999 (local address 127.2.237.2:54723)
0620 22:58:56.959865 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:56.959881 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:56.960053 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:56.960422 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:56.960426 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:56.961160 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:58:56.961678 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:56.961692 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:56.963164 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:56.963174 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:56.964977 (+ 1803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:56.964997 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:56.966010 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:56.966017 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:56.966212 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:56.967069 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:58:56.967086 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:58:56.967957 (+ 871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:58:56.972614 (+ 4657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:56.972618 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:56.972620 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:56.973212 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:56.973705 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:56.973711 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:56.973716 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:56.973853 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:56.974693 (+ 840us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:58:56.974700 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:58:56.974974 (+ 274us) client_negotiation.cc:769] Sending connection context
0620 22:58:56.975176 (+ 202us) client_negotiation.cc:240] Negotiation successful
0620 22:58:56.975623 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":346,"threads_started":1}
I20260620 22:58:56.983186 4942 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:33969
I20260620 22:58:56.983219 5076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:33969 every 8 connection(s)
I20260620 22:58:56.984459 4942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 22:58:56.986653 5077 heartbeater.cc:347] Connected to a master server at 127.2.237.62:37999
I20260620 22:58:56.986882 5077 heartbeater.cc:464] Registering TS with master...
I20260620 22:58:56.987524 5077 heartbeater.cc:511] Master 127.2.237.62:37999 requested a full tablet report, sending...
I20260620 22:58:56.988799 4731 ts_manager.cc:194] Registered new tserver with Master: a0683d16ce0b41fab23ab9d548a9509b (127.2.237.2:33969)
I20260620 22:58:56.989672 4731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.2@KRBTEST.COM'} at 127.2.237.2:38441
I20260620 22:58:56.991621 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 4942
I20260620 22:58:56.991807 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance
I20260620 22:58:56.992247 2996 external_mini_cluster.cc:1428] Setting key 5771c6df2af245ca78084b63e2e2e965
WARNING: no policy specified for kudu/127.2.237.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
I20260620 22:58:57.031045 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:0
--local_ip_for_outbound_sockets=127.2.237.3
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:37999
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:41607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:58:57.348268 5081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:58:57.348564 5081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:58:57.348665 5081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:58:57.356128 5081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:58:57.356271 5081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:58:57.356385 5081 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260620 22:58:57.356446 5081 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 22:58:57.356508 5081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 22:58:57.366080 5081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:41607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.237.62:37999
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:58:57.367362 5081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:58:57.368649 5081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:58:57.381810 5086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:57.381712 5087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:58:57.381739 5089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:58:57.382663 5081 server_base.cc:1034] running on GCE node
I20260620 22:58:57.383266 5081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:58:57.384194 5081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:58:57.385421 5081 hybrid_clock.cc:648] HybridClock initialized: now 1781996337385375 us; error 57 us; skew 500 ppm
I20260620 22:58:57.385722 5081 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:58:57 dist-test-slave-f63b krb5kdc[4686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996337, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:58:57.388773 5081 init.cc:377] Logged in from keytab as kudu/127.2.237.3@KRBTEST.COM (short username kudu)
I20260620 22:58:57.390300 5081 webserver.cc:460] Webserver started at http://127.2.237.3:37835/ using document root <none> and password file <none>
I20260620 22:58:57.391032 5081 fs_manager.cc:359] Metadata directory not provided
I20260620 22:58:57.391170 5081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:58:57.391482 5081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:58:57.394500 5081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-2/data/instance:
uuid: "1e70e1ebf0984593bc40ce95bc7d5500"
format_stamp: "Formatted at 2026-06-20 22:58:57 on dist-test-slave-f63b"
server_key: "50ca4ad57006d98453be44fecec10552"
server_key_iv: "e66aa438a8f5830fa242e74a85d02e39"
server_key_version: "encryptionkey@0"
I20260620 22:58:57.395306 5081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance:
uuid: "1e70e1ebf0984593bc40ce95bc7d5500"
format_stamp: "Formatted at 2026-06-20 22:58:57 on dist-test-slave-f63b"
server_key: "50ca4ad57006d98453be44fecec10552"
server_key_iv: "e66aa438a8f5830fa242e74a85d02e39"
server_key_version: "encryptionkey@0"
I20260620 22:58:57.400631 5081 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260620 22:58:57.404019 5096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:57.405500 5081 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 22:58:57.405658 5081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "1e70e1ebf0984593bc40ce95bc7d5500"
format_stamp: "Formatted at 2026-06-20 22:58:57 on dist-test-slave-f63b"
server_key: "50ca4ad57006d98453be44fecec10552"
server_key_iv: "e66aa438a8f5830fa242e74a85d02e39"
server_key_version: "encryptionkey@0"
I20260620 22:58:57.405910 5081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:58:57.429064 5081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:58:57.433156 5081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:58:57.433461 5081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:58:57.436285 5081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:58:57.436388 5081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:58:57.436547 5081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:58:57.436671 5081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 22:58:57 dist-test-slave-f63b krb5kdc[4686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996337, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:58:57.461148 4826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:57.436826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:36203 (local address 127.2.237.62:37999)
0620 22:58:57.437086 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:57.437095 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:57.438669 (+ 1574us) server_negotiation.cc:407] Connection header received
0620 22:58:57.440750 (+ 2081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:57.440756 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:57.440853 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:57.440959 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:57.443851 (+ 2892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.444820 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.448105 (+ 3285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.448522 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.448625 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:57.454305 (+ 5680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:57.454310 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:57.454313 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:57.454345 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:57.457410 (+ 3065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:57.458514 (+ 1104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:57.458519 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:57.458521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:57.458719 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:57.459360 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:57.459366 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:57.459369 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:57.459678 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:57.459838 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:57.460582 (+ 744us) server_negotiation.cc:299] Negotiation successful
0620 22:58:57.460801 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260620 22:58:57.462152 5105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:57.437537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37999 (local address 127.2.237.3:36203)
0620 22:58:57.438478 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:57.438531 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:57.440429 (+ 1898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:57.441399 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:57.441412 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:57.442519 (+ 1107us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:58:57.443620 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:57.443656 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.445021 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.445027 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:57.447939 (+ 2912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:57.447960 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.448786 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.448795 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:57.449163 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:57.449970 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:58:57.450011 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:58:57.454048 (+ 4037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:58:57.457566 (+ 3518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:57.457580 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:57.457602 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:57.458365 (+ 763us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:57.458849 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:57.458862 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:57.458868 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:57.459208 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:57.459808 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:58:57.459822 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:58:57.460166 (+ 344us) client_negotiation.cc:769] Sending connection context
0620 22:58:57.460425 (+ 259us) client_negotiation.cc:240] Negotiation successful
0620 22:58:57.460970 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":435,"threads_started":1}
I20260620 22:58:57.483040 5161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:57.468038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37999 (local address 127.2.237.3:40381)
0620 22:58:57.468671 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:57.468686 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:57.468831 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:57.469157 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:57.469161 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:57.469607 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:58:57.469868 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:57.469877 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.470878 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.470886 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:57.472766 (+ 1880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:57.472780 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.473508 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.473516 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:57.473678 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:57.474263 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:58:57.474279 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:58:57.475182 (+ 903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:58:57.478757 (+ 3575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:57.478768 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:57.478772 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:57.479547 (+ 775us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:57.480442 (+ 895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:57.480446 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:57.480450 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:57.480590 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:57.481308 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:58:57.481317 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:58:57.481586 (+ 269us) client_negotiation.cc:769] Sending connection context
0620 22:58:57.481757 (+ 171us) client_negotiation.cc:240] Negotiation successful
0620 22:58:57.482226 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":375,"threads_started":1}
I20260620 22:58:57.484369 4826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:57.467958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:40381 (local address 127.2.237.62:37999)
0620 22:58:57.468107 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:57.468111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:57.468770 (+ 659us) server_negotiation.cc:407] Connection header received
0620 22:58:57.468967 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:57.468971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:57.469035 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:57.469136 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:57.470005 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.470741 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.472906 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.473312 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.473453 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:57.475431 (+ 1978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:57.475436 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:57.475443 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:57.475476 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:57.478549 (+ 3073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:57.479733 (+ 1184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:57.479738 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:57.479742 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:57.479901 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:57.480722 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:57.480728 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:57.480734 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:57.481135 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:57.483399 (+ 2264us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:57.483553 (+ 154us) server_negotiation.cc:299] Negotiation successful
0620 22:58:57.483853 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260620 22:58:57.489142 4938 heartbeater.cc:503] Master 127.2.237.62:37999 was elected leader, sending a full tablet report...
I20260620 22:58:57.494906 5215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:40811 every 8 connection(s)
I20260620 22:58:57.494884 5081 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:40811
I20260620 22:58:57.496161 5081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 22:58:57.498034 5216 heartbeater.cc:347] Connected to a master server at 127.2.237.62:37999
I20260620 22:58:57.498243 5216 heartbeater.cc:464] Registering TS with master...
I20260620 22:58:57.498639 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 5081
I20260620 22:58:57.498806 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance
I20260620 22:58:57.499099 5216 heartbeater.cc:511] Master 127.2.237.62:37999 requested a full tablet report, sending...
I20260620 22:58:57.499334 2996 external_mini_cluster.cc:1428] Setting key 7ae060ff5a2cf3ae79946ed4e4eb2f78
I20260620 22:58:57.500614 4731 ts_manager.cc:194] Registered new tserver with Master: 1e70e1ebf0984593bc40ce95bc7d5500 (127.2.237.3:40811)
I20260620 22:58:57.501480 4731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.3@KRBTEST.COM'} at 127.2.237.3:36203
I20260620 22:58:57.502313 2996 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 22:58:57 dist-test-slave-f63b krb5kdc[4686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 22:58:57 dist-test-slave-f63b krb5kdc[4686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:58:57.546809 4826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:57.534224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43084 (local address 127.2.237.62:37999)
0620 22:58:57.534474 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:57.534482 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:57.534800 (+ 318us) server_negotiation.cc:407] Connection header received
0620 22:58:57.534936 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:57.534939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:57.535014 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:57.535099 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:57.536415 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.537136 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.538272 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.538751 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.538845 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:57.542374 (+ 3529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:57.542379 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:57.542382 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:57.542411 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:57.544678 (+ 2267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:57.545250 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:57.545253 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:57.545255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:57.545369 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:57.545659 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:57.545662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:57.545664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:57.545902 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:57.546096 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:57.546270 (+ 174us) server_negotiation.cc:299] Negotiation successful
0620 22:58:57.546420 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260620 22:58:57.550853 4730 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43084:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 22:58:57.557955 4730 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 22:58:57.591214 5231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:57.571179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:33969 (local address 127.0.0.1:36854)
0620 22:58:57.583102 (+ 11923us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:57.583122 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:57.583247 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:57.584025 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:57.584032 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:57.584060 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:58:57.584585 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:57.584604 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.586498 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.586504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:57.588573 (+ 2069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:57.588584 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.590123 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.590129 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:57.590314 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:57.590366 (+ 52us) client_negotiation.cc:769] Sending connection context
0620 22:58:57.590507 (+ 141us) client_negotiation.cc:240] Negotiation successful
0620 22:58:57.590726 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11792,"mutex_wait_us":79,"thread_start_us":10413,"threads_started":1}
I20260620 22:58:57.591683 5234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:57.571743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36854 (local address 127.2.237.2:33969)
0620 22:58:57.583111 (+ 11368us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:57.583124 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:57.583371 (+ 247us) server_negotiation.cc:407] Connection header received
0620 22:58:57.583482 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:57.583489 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:57.583795 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:57.584005 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:58:57.584827 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.586347 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.588732 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.589959 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.590087 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:57.590308 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:57.591233 (+ 925us) server_negotiation.cc:299] Negotiation successful
0620 22:58:57.591403 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10784,"thread_start_us":10645,"threads_started":1}
I20260620 22:58:57.591787 5229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:57.569399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:33195 (local address 127.0.0.1:40742)
0620 22:58:57.570713 (+ 1314us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:57.570752 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:57.570897 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:57.584406 (+ 13509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:57.584412 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:57.584466 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:58:57.584986 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:57.585000 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.586665 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.586669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:57.589138 (+ 2469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:57.589152 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.590978 (+ 1826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.590984 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:57.591147 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:57.591190 (+ 43us) client_negotiation.cc:769] Sending connection context
0620 22:58:57.591279 (+ 89us) client_negotiation.cc:240] Negotiation successful
0620 22:58:57.591412 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1073,"mutex_wait_us":159,"spinlock_wait_cycles":7040,"thread_start_us":1029,"threads_started":1}
I20260620 22:58:57.592721 5230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:57.569625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40742 (local address 127.2.237.1:33195)
0620 22:58:57.583825 (+ 14200us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:57.583830 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:57.583853 (+ 23us) server_negotiation.cc:407] Connection header received
0620 22:58:57.583955 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:57.583964 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:57.584202 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:57.584419 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:58:57.585182 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.586468 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.589293 (+ 2825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.590792 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.590933 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:57.591125 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:57.592288 (+ 1163us) server_negotiation.cc:299] Negotiation successful
0620 22:58:57.592465 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14066,"thread_start_us":13935,"threads_started":1}
I20260620 22:58:57.592820 5233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:57.570181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:40811 (local address 127.0.0.1:56960)
0620 22:58:57.581952 (+ 11771us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:57.581976 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:57.582123 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:57.584265 (+ 2142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:57.584270 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:57.584301 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:58:57.584870 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:57.584885 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.587140 (+ 2255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.587145 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:57.589449 (+ 2304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:57.589458 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.592040 (+ 2582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.592047 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:57.592269 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:57.592300 (+ 31us) client_negotiation.cc:769] Sending connection context
0620 22:58:57.592376 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 22:58:57.592519 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11569,"mutex_wait_us":105,"thread_start_us":12642,"threads_started":1}
I20260620 22:58:57.593617 5232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:57.570560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56960 (local address 127.2.237.3:40811)
0620 22:58:57.583491 (+ 12931us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:57.583499 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:57.583537 (+ 38us) server_negotiation.cc:407] Connection header received
0620 22:58:57.583610 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:57.583615 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:57.583806 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:57.584005 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:58:57.585079 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.587012 (+ 1933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.589587 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.591101 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.591231 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:57.593058 (+ 1827us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:57.593134 (+ 76us) server_negotiation.cc:299] Negotiation successful
0620 22:58:57.593302 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12738,"thread_start_us":12571,"threads_started":1}
I20260620 22:58:57.596105 5007 tablet_service.cc:1467] Processing CreateTablet for tablet 57a68056240948c98c4dbbccb5fcc382 (DEFAULT_TABLE table=test-table [id=d5e317299e5f4b5cb00b55de8be1d969]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:58:57.596475 5146 tablet_service.cc:1467] Processing CreateTablet for tablet 57a68056240948c98c4dbbccb5fcc382 (DEFAULT_TABLE table=test-table [id=d5e317299e5f4b5cb00b55de8be1d969]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:58:57.596885 4868 tablet_service.cc:1467] Processing CreateTablet for tablet 57a68056240948c98c4dbbccb5fcc382 (DEFAULT_TABLE table=test-table [id=d5e317299e5f4b5cb00b55de8be1d969]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:58:57.597544 5007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57a68056240948c98c4dbbccb5fcc382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:58:57.597612 5146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57a68056240948c98c4dbbccb5fcc382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:58:57.598062 4868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57a68056240948c98c4dbbccb5fcc382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:58:57.609162 5235 tablet_bootstrap.cc:492] T 57a68056240948c98c4dbbccb5fcc382 P 1e70e1ebf0984593bc40ce95bc7d5500: Bootstrap starting.
I20260620 22:58:57.611817 5237 tablet_bootstrap.cc:492] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938: Bootstrap starting.
I20260620 22:58:57.612519 5236 tablet_bootstrap.cc:492] T 57a68056240948c98c4dbbccb5fcc382 P a0683d16ce0b41fab23ab9d548a9509b: Bootstrap starting.
I20260620 22:58:57.612900 5235 tablet_bootstrap.cc:654] T 57a68056240948c98c4dbbccb5fcc382 P 1e70e1ebf0984593bc40ce95bc7d5500: Neither blocks nor log segments found. Creating new log.
I20260620 22:58:57.614176 5235 log.cc:824] T 57a68056240948c98c4dbbccb5fcc382 P 1e70e1ebf0984593bc40ce95bc7d5500: Log is configured to *not* fsync() on all Append() calls
I20260620 22:58:57.616150 5237 tablet_bootstrap.cc:654] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938: Neither blocks nor log segments found. Creating new log.
I20260620 22:58:57.617107 5236 tablet_bootstrap.cc:654] T 57a68056240948c98c4dbbccb5fcc382 P a0683d16ce0b41fab23ab9d548a9509b: Neither blocks nor log segments found. Creating new log.
I20260620 22:58:57.617360 5235 tablet_bootstrap.cc:492] T 57a68056240948c98c4dbbccb5fcc382 P 1e70e1ebf0984593bc40ce95bc7d5500: No bootstrap required, opened a new log
I20260620 22:58:57.617355 5237 log.cc:824] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938: Log is configured to *not* fsync() on all Append() calls
I20260620 22:58:57.617630 5235 ts_tablet_manager.cc:1397] T 57a68056240948c98c4dbbccb5fcc382 P 1e70e1ebf0984593bc40ce95bc7d5500: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260620 22:58:57.618289 5236 log.cc:824] T 57a68056240948c98c4dbbccb5fcc382 P a0683d16ce0b41fab23ab9d548a9509b: Log is configured to *not* fsync() on all Append() calls
I20260620 22:58:57.620805 5236 tablet_bootstrap.cc:492] T 57a68056240948c98c4dbbccb5fcc382 P a0683d16ce0b41fab23ab9d548a9509b: No bootstrap required, opened a new log
I20260620 22:58:57.620820 5237 tablet_bootstrap.cc:492] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938: No bootstrap required, opened a new log
I20260620 22:58:57.621069 5236 ts_tablet_manager.cc:1397] T 57a68056240948c98c4dbbccb5fcc382 P a0683d16ce0b41fab23ab9d548a9509b: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260620 22:58:57.621074 5237 ts_tablet_manager.cc:1397] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260620 22:58:57.627692 5236 raft_consensus.cc:348] T 57a68056240948c98c4dbbccb5fcc382 P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:58:57.628058 5236 raft_consensus.cc:374] T 57a68056240948c98c4dbbccb5fcc382 P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:58:57.628178 5236 raft_consensus.cc:729] T 57a68056240948c98c4dbbccb5fcc382 P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0683d16ce0b41fab23ab9d548a9509b, State: Initialized, Role: FOLLOWER
I20260620 22:58:57.628297 5235 raft_consensus.cc:348] T 57a68056240948c98c4dbbccb5fcc382 P 1e70e1ebf0984593bc40ce95bc7d5500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:58:57.628731 5235 raft_consensus.cc:374] T 57a68056240948c98c4dbbccb5fcc382 P 1e70e1ebf0984593bc40ce95bc7d5500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:58:57.628821 5235 raft_consensus.cc:729] T 57a68056240948c98c4dbbccb5fcc382 P 1e70e1ebf0984593bc40ce95bc7d5500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e70e1ebf0984593bc40ce95bc7d5500, State: Initialized, Role: FOLLOWER
I20260620 22:58:57.628865 5236 consensus_queue.cc:260] T 57a68056240948c98c4dbbccb5fcc382 P a0683d16ce0b41fab23ab9d548a9509b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:58:57.629370 5235 consensus_queue.cc:260] T 57a68056240948c98c4dbbccb5fcc382 P 1e70e1ebf0984593bc40ce95bc7d5500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:58:57.630033 5077 heartbeater.cc:503] Master 127.2.237.62:37999 was elected leader, sending a full tablet report...
I20260620 22:58:57.630728 5236 ts_tablet_manager.cc:1428] T 57a68056240948c98c4dbbccb5fcc382 P a0683d16ce0b41fab23ab9d548a9509b: Time spent starting tablet: real 0.010s user 0.010s sys 0.001s
I20260620 22:58:57.631603 5237 raft_consensus.cc:348] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:58:57.632167 5237 raft_consensus.cc:374] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:58:57.632337 5237 raft_consensus.cc:729] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f832345cd08e4521bff81fe662f9e938, State: Initialized, Role: FOLLOWER
I20260620 22:58:57.633025 5237 consensus_queue.cc:260] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:58:57.635545 5237 ts_tablet_manager.cc:1428] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260620 22:58:57.636785 5216 heartbeater.cc:503] Master 127.2.237.62:37999 was elected leader, sending a full tablet report...
I20260620 22:58:57.637533 5235 ts_tablet_manager.cc:1428] T 57a68056240948c98c4dbbccb5fcc382 P 1e70e1ebf0984593bc40ce95bc7d5500: Time spent starting tablet: real 0.020s user 0.016s sys 0.004s
W20260620 22:58:57.732864 4939 tablet.cc:2369] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 22:58:57.733548 5243 raft_consensus.cc:482] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 22:58:57.733847 5243 raft_consensus.cc:504] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:58:57.735481 5243 leader_election.cc:288] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0683d16ce0b41fab23ab9d548a9509b (127.2.237.2:33969), 1e70e1ebf0984593bc40ce95bc7d5500 (127.2.237.3:40811)
W20260620 22:58:57.736449 5078 tablet.cc:2369] T 57a68056240948c98c4dbbccb5fcc382 P a0683d16ce0b41fab23ab9d548a9509b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 22:58:57.742280 5232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:57.735994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:41105 (local address 127.2.237.3:40811)
0620 22:58:57.736201 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:57.736208 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:57.736890 (+ 682us) server_negotiation.cc:407] Connection header received
0620 22:58:57.737141 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:57.737148 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:57.737223 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:57.737308 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:58:57.737962 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.738727 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.740596 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.741343 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.741504 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:57.741613 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:57.741894 (+ 281us) server_negotiation.cc:299] Negotiation successful
0620 22:58:57.742020 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260620 22:58:57.742347 5234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:57.736164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:44543 (local address 127.2.237.2:33969)
0620 22:58:57.736364 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:57.736372 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:57.736880 (+ 508us) server_negotiation.cc:407] Connection header received
0620 22:58:57.737097 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:57.737102 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:57.737191 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:57.737303 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:58:57.737948 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.738714 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.740631 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.741380 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.741537 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:57.741647 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:57.741940 (+ 293us) server_negotiation.cc:299] Negotiation successful
0620 22:58:57.742057 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260620 22:58:57.742449 5244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:57.735911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:40811 (local address 127.2.237.1:41105)
0620 22:58:57.736797 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:57.736814 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:57.736982 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:57.737406 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:57.737411 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:57.737438 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:58:57.737803 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:57.737811 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.738857 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.738862 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:57.740445 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:57.740457 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.741570 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.741577 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:57.741739 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:57.741761 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 22:58:57.741869 (+ 108us) client_negotiation.cc:240] Negotiation successful
0620 22:58:57.742007 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"mutex_wait_us":21,"thread_start_us":434,"threads_started":1}
I20260620 22:58:57.742513 5245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:57.735786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:33969 (local address 127.2.237.1:44543)
0620 22:58:57.736737 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:57.736757 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:57.736939 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:57.737371 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:57.737376 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:57.737403 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:58:57.737794 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:57.737805 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.738837 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.738845 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:57.740477 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:57.740489 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.741609 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.741615 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:57.741778 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:57.741800 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 22:58:57.741903 (+ 103us) client_negotiation.cc:240] Negotiation successful
0620 22:58:57.742022 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"mutex_wait_us":60,"thread_start_us":1396,"threads_started":1}
I20260620 22:58:57.743520 5031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57a68056240948c98c4dbbccb5fcc382" candidate_uuid: "f832345cd08e4521bff81fe662f9e938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0683d16ce0b41fab23ab9d548a9509b" is_pre_election: true
I20260620 22:58:57.744024 5031 raft_consensus.cc:2454] T 57a68056240948c98c4dbbccb5fcc382 P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f832345cd08e4521bff81fe662f9e938 in term 0.
I20260620 22:58:57.744653 4819 leader_election.cc:302] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0683d16ce0b41fab23ab9d548a9509b, f832345cd08e4521bff81fe662f9e938; no voters:
I20260620 22:58:57.744805 5171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57a68056240948c98c4dbbccb5fcc382" candidate_uuid: "f832345cd08e4521bff81fe662f9e938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" is_pre_election: true
I20260620 22:58:57.745054 5243 raft_consensus.cc:2781] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 22:58:57.745214 5243 raft_consensus.cc:482] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 22:58:57.745304 5243 raft_consensus.cc:3037] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:58:57.745394 5171 raft_consensus.cc:2454] T 57a68056240948c98c4dbbccb5fcc382 P 1e70e1ebf0984593bc40ce95bc7d5500 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f832345cd08e4521bff81fe662f9e938 in term 0.
W20260620 22:58:57.747663 5217 tablet.cc:2369] T 57a68056240948c98c4dbbccb5fcc382 P 1e70e1ebf0984593bc40ce95bc7d5500: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 22:58:57.747675 5243 raft_consensus.cc:504] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:58:57.748373 5243 leader_election.cc:288] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938 [CANDIDATE]: Term 1 election: Requested vote from peers a0683d16ce0b41fab23ab9d548a9509b (127.2.237.2:33969), 1e70e1ebf0984593bc40ce95bc7d5500 (127.2.237.3:40811)
I20260620 22:58:57.748801 5171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57a68056240948c98c4dbbccb5fcc382" candidate_uuid: "f832345cd08e4521bff81fe662f9e938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e70e1ebf0984593bc40ce95bc7d5500"
I20260620 22:58:57.748786 5031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57a68056240948c98c4dbbccb5fcc382" candidate_uuid: "f832345cd08e4521bff81fe662f9e938" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0683d16ce0b41fab23ab9d548a9509b"
I20260620 22:58:57.748987 5171 raft_consensus.cc:3037] T 57a68056240948c98c4dbbccb5fcc382 P 1e70e1ebf0984593bc40ce95bc7d5500 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:58:57.748987 5031 raft_consensus.cc:3037] T 57a68056240948c98c4dbbccb5fcc382 P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:58:57.752138 5031 raft_consensus.cc:2454] T 57a68056240948c98c4dbbccb5fcc382 P a0683d16ce0b41fab23ab9d548a9509b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f832345cd08e4521bff81fe662f9e938 in term 1.
I20260620 22:58:57.752137 5171 raft_consensus.cc:2454] T 57a68056240948c98c4dbbccb5fcc382 P 1e70e1ebf0984593bc40ce95bc7d5500 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f832345cd08e4521bff81fe662f9e938 in term 1.
I20260620 22:58:57.752530 4820 leader_election.cc:302] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e70e1ebf0984593bc40ce95bc7d5500, f832345cd08e4521bff81fe662f9e938; no voters:
I20260620 22:58:57.752848 5243 raft_consensus.cc:2781] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:58:57.753227 5243 raft_consensus.cc:686] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938 [term 1 LEADER]: Becoming Leader. State: Replica: f832345cd08e4521bff81fe662f9e938, State: Running, Role: LEADER
I20260620 22:58:57.753775 5243 consensus_queue.cc:237] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:58:57.758436 4731 catalog_manager.cc:5495] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938 reported cstate change: term changed from 0 to 1, leader changed from <none> to f832345cd08e4521bff81fe662f9e938 (127.2.237.1). New cstate: current_term: 1 leader_uuid: "f832345cd08e4521bff81fe662f9e938" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } health_report { overall_health: UNKNOWN } } }
Jun 20 22:58:57 dist-test-slave-f63b krb5kdc[4686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996335, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:58:57.838073 4826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:57.824825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43088 (local address 127.2.237.62:37999)
0620 22:58:57.824997 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:57.825002 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:57.825536 (+ 534us) server_negotiation.cc:407] Connection header received
0620 22:58:57.825731 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:57.825742 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:57.825793 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:57.825894 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:58:57.827083 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.827822 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.829093 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.829493 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.829635 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:57.832986 (+ 3351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:58:57.832991 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:58:57.832994 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:58:57.833019 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:58:57.835485 (+ 2466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:57.836151 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:57.836157 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:57.836163 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:57.836275 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:58:57.836638 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:58:57.836643 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:58:57.836647 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:58:57.836900 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:58:57.837065 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:57.837439 (+ 374us) server_negotiation.cc:299] Negotiation successful
0620 22:58:57.837684 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":56}
I20260620 22:58:57.838199 5253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:57.824867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37999 (local address 127.0.0.1:43088)
0620 22:58:57.825426 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:57.825441 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:57.825613 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:57.825912 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:57.825916 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:57.826681 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:58:57.826936 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:57.826946 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.827961 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.827969 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:57.828951 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:57.828962 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.829679 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.829685 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:57.829891 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:57.830539 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:58:57.830559 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:58:57.832786 (+ 2227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:58:57.835625 (+ 2839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:57.835637 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:57.835645 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:57.836034 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:57.836386 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:58:57.836391 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:58:57.836395 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:58:57.836526 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:58:57.837032 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:58:57.837040 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:58:57.837209 (+ 169us) client_negotiation.cc:769] Sending connection context
0620 22:58:57.837365 (+ 156us) client_negotiation.cc:240] Negotiation successful
0620 22:58:57.837798 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":168,"thread_start_us":316,"threads_started":1}
I20260620 22:58:57.843246 4731 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:43088:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260620 22:58:57.844296 4731 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 22:58:57.850589 4868 tablet_service.cc:1467] Processing CreateTablet for tablet 0b11d0473a124f93a0bd777a96bd75fd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bc788daaa1e14606967cf2f3c03a6a5f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 22:58:57.850983 5146 tablet_service.cc:1467] Processing CreateTablet for tablet 0b11d0473a124f93a0bd777a96bd75fd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bc788daaa1e14606967cf2f3c03a6a5f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 22:58:57.851658 4868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b11d0473a124f93a0bd777a96bd75fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:58:57.851799 5146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b11d0473a124f93a0bd777a96bd75fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:58:57.855000 5007 tablet_service.cc:1467] Processing CreateTablet for tablet 0b11d0473a124f93a0bd777a96bd75fd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bc788daaa1e14606967cf2f3c03a6a5f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 22:58:57.855831 5007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b11d0473a124f93a0bd777a96bd75fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:58:57.858640 5235 tablet_bootstrap.cc:492] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500: Bootstrap starting.
I20260620 22:58:57.860229 5235 tablet_bootstrap.cc:654] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500: Neither blocks nor log segments found. Creating new log.
I20260620 22:58:57.861824 5236 tablet_bootstrap.cc:492] T 0b11d0473a124f93a0bd777a96bd75fd P a0683d16ce0b41fab23ab9d548a9509b: Bootstrap starting.
I20260620 22:58:57.862676 5237 tablet_bootstrap.cc:492] T 0b11d0473a124f93a0bd777a96bd75fd P f832345cd08e4521bff81fe662f9e938: Bootstrap starting.
I20260620 22:58:57.864107 5236 tablet_bootstrap.cc:654] T 0b11d0473a124f93a0bd777a96bd75fd P a0683d16ce0b41fab23ab9d548a9509b: Neither blocks nor log segments found. Creating new log.
I20260620 22:58:57.864230 5237 tablet_bootstrap.cc:654] T 0b11d0473a124f93a0bd777a96bd75fd P f832345cd08e4521bff81fe662f9e938: Neither blocks nor log segments found. Creating new log.
I20260620 22:58:57.872148 5237 tablet_bootstrap.cc:492] T 0b11d0473a124f93a0bd777a96bd75fd P f832345cd08e4521bff81fe662f9e938: No bootstrap required, opened a new log
I20260620 22:58:57.872308 5237 ts_tablet_manager.cc:1397] T 0b11d0473a124f93a0bd777a96bd75fd P f832345cd08e4521bff81fe662f9e938: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260620 22:58:57.872494 5235 tablet_bootstrap.cc:492] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500: No bootstrap required, opened a new log
I20260620 22:58:57.872859 5235 ts_tablet_manager.cc:1397] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.008s
I20260620 22:58:57.873090 5236 tablet_bootstrap.cc:492] T 0b11d0473a124f93a0bd777a96bd75fd P a0683d16ce0b41fab23ab9d548a9509b: No bootstrap required, opened a new log
I20260620 22:58:57.873284 5236 ts_tablet_manager.cc:1397] T 0b11d0473a124f93a0bd777a96bd75fd P a0683d16ce0b41fab23ab9d548a9509b: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.002s
I20260620 22:58:57.873226 5237 raft_consensus.cc:348] T 0b11d0473a124f93a0bd777a96bd75fd P f832345cd08e4521bff81fe662f9e938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:58:57.873536 5237 raft_consensus.cc:374] T 0b11d0473a124f93a0bd777a96bd75fd P f832345cd08e4521bff81fe662f9e938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:58:57.873682 5237 raft_consensus.cc:729] T 0b11d0473a124f93a0bd777a96bd75fd P f832345cd08e4521bff81fe662f9e938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f832345cd08e4521bff81fe662f9e938, State: Initialized, Role: FOLLOWER
I20260620 22:58:57.874013 5237 consensus_queue.cc:260] T 0b11d0473a124f93a0bd777a96bd75fd P f832345cd08e4521bff81fe662f9e938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:58:57.874013 5236 raft_consensus.cc:348] T 0b11d0473a124f93a0bd777a96bd75fd P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:58:57.873700 5235 raft_consensus.cc:348] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:58:57.874238 5236 raft_consensus.cc:374] T 0b11d0473a124f93a0bd777a96bd75fd P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:58:57.874308 5235 raft_consensus.cc:374] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:58:57.874354 5236 raft_consensus.cc:729] T 0b11d0473a124f93a0bd777a96bd75fd P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0683d16ce0b41fab23ab9d548a9509b, State: Initialized, Role: FOLLOWER
I20260620 22:58:57.874447 5235 raft_consensus.cc:729] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e70e1ebf0984593bc40ce95bc7d5500, State: Initialized, Role: FOLLOWER
I20260620 22:58:57.874552 5243 tablet_replica.cc:439] T 0b11d0473a124f93a0bd777a96bd75fd P f832345cd08e4521bff81fe662f9e938: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } } }
I20260620 22:58:57.874624 5236 consensus_queue.cc:260] T 0b11d0473a124f93a0bd777a96bd75fd P a0683d16ce0b41fab23ab9d548a9509b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:58:57.874866 5243 tablet_replica.cc:442] T 0b11d0473a124f93a0bd777a96bd75fd P f832345cd08e4521bff81fe662f9e938: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 22:58:57.874753 5235 consensus_queue.cc:260] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:58:57.875051 5241 tablet_replica.cc:439] T 0b11d0473a124f93a0bd777a96bd75fd P a0683d16ce0b41fab23ab9d548a9509b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } } }
I20260620 22:58:57.875314 5235 ts_tablet_manager.cc:1428] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260620 22:58:57.875394 5241 tablet_replica.cc:442] T 0b11d0473a124f93a0bd777a96bd75fd P a0683d16ce0b41fab23ab9d548a9509b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 22:58:57.875746 5242 tablet_replica.cc:439] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } } }
I20260620 22:58:57.876101 5242 tablet_replica.cc:442] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 22:58:57.876824 5237 ts_tablet_manager.cc:1428] T 0b11d0473a124f93a0bd777a96bd75fd P f832345cd08e4521bff81fe662f9e938: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260620 22:58:57.876837 5236 ts_tablet_manager.cc:1428] T 0b11d0473a124f93a0bd777a96bd75fd P a0683d16ce0b41fab23ab9d548a9509b: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260620 22:58:57.897118 5242 raft_consensus.cc:482] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 22:58:57.897393 5242 raft_consensus.cc:504] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:58:57.899042 5242 leader_election.cc:288] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f832345cd08e4521bff81fe662f9e938 (127.2.237.1:33195), a0683d16ce0b41fab23ab9d548a9509b (127.2.237.2:33969)
I20260620 22:58:57.904685 5105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:57.899230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:33195 (local address 127.2.237.3:41167)
0620 22:58:57.899449 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:57.899470 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:57.899594 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:57.900012 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:57.900017 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:57.900048 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:58:57.900230 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:57.900238 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.901200 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.901205 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:57.902457 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:57.902468 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.903938 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.903944 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:57.904132 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:57.904157 (+ 25us) client_negotiation.cc:769] Sending connection context
0620 22:58:57.904263 (+ 106us) client_negotiation.cc:240] Negotiation successful
0620 22:58:57.904391 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260620 22:58:57.904667 5230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:57.899342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:41167 (local address 127.2.237.1:33195)
0620 22:58:57.899557 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:57.899563 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:57.899582 (+ 19us) server_negotiation.cc:407] Connection header received
0620 22:58:57.899745 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:57.899751 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:57.899849 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:57.899935 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:58:57.900365 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.901065 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.902643 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.903777 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.903911 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:57.904011 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:57.904288 (+ 277us) server_negotiation.cc:299] Negotiation successful
0620 22:58:57.904410 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260620 22:58:57.905649 4892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b11d0473a124f93a0bd777a96bd75fd" candidate_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f832345cd08e4521bff81fe662f9e938" is_pre_election: true
I20260620 22:58:57.905882 5234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:57.899716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:56533 (local address 127.2.237.2:33969)
0620 22:58:57.899910 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:57.899931 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:57.900374 (+ 443us) server_negotiation.cc:407] Connection header received
0620 22:58:57.900577 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:57.900586 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:57.900639 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:57.900734 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:58:57.901222 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.901962 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.904004 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.904901 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.905033 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:57.905141 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:57.905477 (+ 336us) server_negotiation.cc:299] Negotiation successful
0620 22:58:57.905600 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260620 22:58:57.906016 5257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:57.899599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:33969 (local address 127.2.237.3:56533)
0620 22:58:57.900268 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:57.900290 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:57.900437 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:57.900746 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:57.900750 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:57.900772 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:58:57.901088 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:57.901096 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.902092 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.902098 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:57.903870 (+ 1772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:57.903881 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.905191 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.905198 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:57.905323 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:57.905345 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 22:58:57.905443 (+ 98us) client_negotiation.cc:240] Negotiation successful
0620 22:58:57.905561 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":358,"threads_started":1}
I20260620 22:58:57.906104 4892 raft_consensus.cc:2454] T 0b11d0473a124f93a0bd777a96bd75fd P f832345cd08e4521bff81fe662f9e938 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e70e1ebf0984593bc40ce95bc7d5500 in term 0.
I20260620 22:58:57.906814 5098 leader_election.cc:302] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e70e1ebf0984593bc40ce95bc7d5500, f832345cd08e4521bff81fe662f9e938; no voters:
I20260620 22:58:57.906978 5031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b11d0473a124f93a0bd777a96bd75fd" candidate_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0683d16ce0b41fab23ab9d548a9509b" is_pre_election: true
I20260620 22:58:57.907228 5031 raft_consensus.cc:2454] T 0b11d0473a124f93a0bd777a96bd75fd P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e70e1ebf0984593bc40ce95bc7d5500 in term 0.
I20260620 22:58:57.907503 5242 raft_consensus.cc:2781] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 22:58:57.907626 5242 raft_consensus.cc:482] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 22:58:57.907727 5242 raft_consensus.cc:3037] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:58:57.910107 5242 raft_consensus.cc:504] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:58:57.910774 5242 leader_election.cc:288] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500 [CANDIDATE]: Term 1 election: Requested vote from peers f832345cd08e4521bff81fe662f9e938 (127.2.237.1:33195), a0683d16ce0b41fab23ab9d548a9509b (127.2.237.2:33969)
I20260620 22:58:57.911195 4892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b11d0473a124f93a0bd777a96bd75fd" candidate_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f832345cd08e4521bff81fe662f9e938"
I20260620 22:58:57.911232 5031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b11d0473a124f93a0bd777a96bd75fd" candidate_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0683d16ce0b41fab23ab9d548a9509b"
I20260620 22:58:57.911401 4892 raft_consensus.cc:3037] T 0b11d0473a124f93a0bd777a96bd75fd P f832345cd08e4521bff81fe662f9e938 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:58:57.911440 5031 raft_consensus.cc:3037] T 0b11d0473a124f93a0bd777a96bd75fd P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:58:57.914330 5031 raft_consensus.cc:2454] T 0b11d0473a124f93a0bd777a96bd75fd P a0683d16ce0b41fab23ab9d548a9509b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e70e1ebf0984593bc40ce95bc7d5500 in term 1.
I20260620 22:58:57.914388 4892 raft_consensus.cc:2454] T 0b11d0473a124f93a0bd777a96bd75fd P f832345cd08e4521bff81fe662f9e938 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e70e1ebf0984593bc40ce95bc7d5500 in term 1.
I20260620 22:58:57.914718 5098 leader_election.cc:302] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e70e1ebf0984593bc40ce95bc7d5500, f832345cd08e4521bff81fe662f9e938; no voters:
I20260620 22:58:57.915198 5242 raft_consensus.cc:2781] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:58:57.915688 5242 raft_consensus.cc:686] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500 [term 1 LEADER]: Becoming Leader. State: Replica: 1e70e1ebf0984593bc40ce95bc7d5500, State: Running, Role: LEADER
I20260620 22:58:57.916465 5242 consensus_queue.cc:237] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:58:57.919432 5258 tablet_replica.cc:439] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500: TxnStatusTablet state changed. Reason: New leader 1e70e1ebf0984593bc40ce95bc7d5500. Latest consensus state: current_term: 1 leader_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } } }
I20260620 22:58:57.919800 5258 tablet_replica.cc:442] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500: This TxnStatusTablet replica's current role is: LEADER
I20260620 22:58:57.920596 5260 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260620 22:58:57.920843 5260 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260620 22:58:57.921145 4728 catalog_manager.cc:5495] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e70e1ebf0984593bc40ce95bc7d5500 (127.2.237.3). New cstate: current_term: 1 leader_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } health_report { overall_health: HEALTHY } } }
I20260620 22:58:57.946660 5253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:57.940913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:40811 (local address 127.0.0.1:56972)
0620 22:58:57.941155 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:57.941172 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:57.941343 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:57.941811 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:57.941816 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:57.941828 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 22:58:57.942024 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:57.942032 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.943189 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.943197 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:57.944201 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:57.944212 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.944963 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.944969 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:57.945151 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:57.945206 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 22:58:57.946063 (+ 857us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 22:58:57.946086 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 22:58:57.946175 (+ 89us) client_negotiation.cc:240] Negotiation successful
0620 22:58:57.946339 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260620 22:58:57.946640 5232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:57.940896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56972 (local address 127.2.237.3:40811)
0620 22:58:57.941071 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:57.941078 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:57.941251 (+ 173us) server_negotiation.cc:407] Connection header received
0620 22:58:57.941558 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:57.941562 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:57.941647 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:57.941782 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 22:58:57.942168 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.943061 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.944347 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:57.944795 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:57.944951 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:57.945402 (+ 451us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 22:58:57.945955 (+ 553us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 22:58:57.946116 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:57.946193 (+ 77us) server_negotiation.cc:299] Negotiation successful
0620 22:58:57.946274 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 22:58:57.954730 4892 raft_consensus.cc:1261] T 0b11d0473a124f93a0bd777a96bd75fd P f832345cd08e4521bff81fe662f9e938 [term 1 FOLLOWER]: Refusing update from remote peer 1e70e1ebf0984593bc40ce95bc7d5500: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:58:57.954783 5031 raft_consensus.cc:1261] T 0b11d0473a124f93a0bd777a96bd75fd P a0683d16ce0b41fab23ab9d548a9509b [term 1 FOLLOWER]: Refusing update from remote peer 1e70e1ebf0984593bc40ce95bc7d5500: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:58:57.955760 5258 consensus_queue.cc:1035] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:58:57.956126 5242 consensus_queue.cc:1035] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:58:57.966034 5243 tablet_replica.cc:439] T 0b11d0473a124f93a0bd777a96bd75fd P f832345cd08e4521bff81fe662f9e938: TxnStatusTablet state changed. Reason: New leader 1e70e1ebf0984593bc40ce95bc7d5500. Latest consensus state: current_term: 1 leader_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } } }
I20260620 22:58:57.966503 5243 tablet_replica.cc:442] T 0b11d0473a124f93a0bd777a96bd75fd P f832345cd08e4521bff81fe662f9e938: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 22:58:57.968701 5241 tablet_replica.cc:439] T 0b11d0473a124f93a0bd777a96bd75fd P a0683d16ce0b41fab23ab9d548a9509b: TxnStatusTablet state changed. Reason: New leader 1e70e1ebf0984593bc40ce95bc7d5500. Latest consensus state: current_term: 1 leader_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } } }
I20260620 22:58:57.969002 5241 tablet_replica.cc:442] T 0b11d0473a124f93a0bd777a96bd75fd P a0683d16ce0b41fab23ab9d548a9509b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 22:58:57.972594 5263 mvcc.cc:205] Tried to move back new op lower bound from 7299057000252760064 to 7299057000109596672. Current Snapshot: MvccSnapshot[applied={T|T < 7299057000252760064}]
I20260620 22:58:57.973467 5258 tablet_replica.cc:439] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } } }
I20260620 22:58:57.973754 5258 tablet_replica.cc:442] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500: This TxnStatusTablet replica's current role is: LEADER
I20260620 22:58:57.975384 5242 tablet_replica.cc:439] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } } }
I20260620 22:58:57.975653 5242 tablet_replica.cc:442] T 0b11d0473a124f93a0bd777a96bd75fd P 1e70e1ebf0984593bc40ce95bc7d5500: This TxnStatusTablet replica's current role is: LEADER
I20260620 22:58:57.976161 5264 mvcc.cc:205] Tried to move back new op lower bound from 7299057000252760064 to 7299057000109596672. Current Snapshot: MvccSnapshot[applied={T|T < 7299057000252760064}]
I20260620 22:58:57.980181 5265 mvcc.cc:205] Tried to move back new op lower bound from 7299057000252760064 to 7299057000109596672. Current Snapshot: MvccSnapshot[applied={T|T < 7299057000252760064}]
I20260620 22:58:58.008206 5241 tablet_replica.cc:439] T 0b11d0473a124f93a0bd777a96bd75fd P a0683d16ce0b41fab23ab9d548a9509b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } } }
I20260620 22:58:58.008561 5241 tablet_replica.cc:442] T 0b11d0473a124f93a0bd777a96bd75fd P a0683d16ce0b41fab23ab9d548a9509b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 22:58:58.018926 5230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:58.011001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40748 (local address 127.2.237.1:33195)
0620 22:58:58.011168 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:58.011175 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:58.011194 (+ 19us) server_negotiation.cc:407] Connection header received
0620 22:58:58.011483 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:58.011492 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:58.011608 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:58.011771 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 22:58:58.012363 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:58.013548 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:58.015584 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:58.016246 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:58.016370 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:58.017009 (+ 639us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 22:58:58.017750 (+ 741us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 22:58:58.017887 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:58.018456 (+ 569us) server_negotiation.cc:299] Negotiation successful
0620 22:58:58.018544 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260620 22:58:58.019330 5243 tablet_replica.cc:439] T 0b11d0473a124f93a0bd777a96bd75fd P f832345cd08e4521bff81fe662f9e938: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } } }
I20260620 22:58:58.019625 5243 tablet_replica.cc:442] T 0b11d0473a124f93a0bd777a96bd75fd P f832345cd08e4521bff81fe662f9e938: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 22:58:58.037900 5232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:58.031543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:54141 (local address 127.2.237.3:40811)
0620 22:58:58.031712 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:58.031718 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:58.032346 (+ 628us) server_negotiation.cc:407] Connection header received
0620 22:58:58.032537 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:58.032541 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:58.032603 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:58.032679 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 22:58:58.033249 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:58.034005 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:58.036086 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:58.036533 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:58.036661 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:58.037261 (+ 600us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 22:58:58.037368 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 22:58:58.037451 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:58.037615 (+ 164us) server_negotiation.cc:299] Negotiation successful
0620 22:58:58.037679 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260620 22:58:58.038223 5275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:58.031514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:40811 (local address 127.2.237.1:54141)
0620 22:58:58.032224 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:58.032244 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:58.032396 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:58.032720 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:58.032725 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:58.032738 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 22:58:58.033104 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:58.033115 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:58.034128 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:58.034134 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:58.035924 (+ 1790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:58.035939 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:58.036762 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:58.036782 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:58.036949 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:58.037018 (+ 69us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 22:58:58.037479 (+ 461us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 22:58:58.037498 (+ 19us) client_negotiation.cc:769] Sending connection context
0620 22:58:58.037583 (+ 85us) client_negotiation.cc:240] Negotiation successful
0620 22:58:58.037731 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":327,"threads_started":1}
I20260620 22:58:58.048143 5031 raft_consensus.cc:1261] T 57a68056240948c98c4dbbccb5fcc382 P a0683d16ce0b41fab23ab9d548a9509b [term 1 FOLLOWER]: Refusing update from remote peer f832345cd08e4521bff81fe662f9e938: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:58:58.048219 5171 raft_consensus.cc:1261] T 57a68056240948c98c4dbbccb5fcc382 P 1e70e1ebf0984593bc40ce95bc7d5500 [term 1 FOLLOWER]: Refusing update from remote peer f832345cd08e4521bff81fe662f9e938: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:58:58.048870 5243 consensus_queue.cc:1035] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:58:58.049196 5246 consensus_queue.cc:1035] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:58:58.087744 5230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:58.080572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:39403 (local address 127.2.237.1:33195)
0620 22:58:58.080736 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:58.080742 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:58.081208 (+ 466us) server_negotiation.cc:407] Connection header received
0620 22:58:58.081399 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:58.081405 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:58.081502 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:58.081591 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 22:58:58.082286 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:58.083451 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:58.085607 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:58.086271 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:58.086417 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:58.087051 (+ 634us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 22:58:58.087178 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 22:58:58.087268 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:58.087363 (+ 95us) server_negotiation.cc:299] Negotiation successful
0620 22:58:58.087457 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 22:58:58.087793 5279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:58.080489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:33195 (local address 127.2.237.3:39403)
0620 22:58:58.081111 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:58.081127 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:58.081244 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:58.081746 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:58.081752 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:58.081764 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 22:58:58.082092 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:58.082104 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:58.083636 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:58.083646 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:58.085461 (+ 1815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:58.085476 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:58.086599 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:58.086607 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:58.086779 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:58.086854 (+ 75us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 22:58:58.087275 (+ 421us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 22:58:58.087282 (+ 7us) client_negotiation.cc:769] Sending connection context
0620 22:58:58.087343 (+ 61us) client_negotiation.cc:240] Negotiation successful
0620 22:58:58.087501 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":298,"threads_started":1}
I20260620 22:58:58.121341 4728 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 22:58:58.121629 4728 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 22:58:58.125491 4728 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8ba9bd49121f41eb97acc05b4e83ac70: Sending DeleteTablet for 3 replicas of tablet 57a68056240948c98c4dbbccb5fcc382
I20260620 22:58:58.126696 5007 tablet_service.cc:1514] Processing DeleteTablet for tablet 57a68056240948c98c4dbbccb5fcc382 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:58:58 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:36854
I20260620 22:58:58.126780 5146 tablet_service.cc:1514] Processing DeleteTablet for tablet 57a68056240948c98c4dbbccb5fcc382 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:58:58 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:56960
I20260620 22:58:58.127816 4868 tablet_service.cc:1514] Processing DeleteTablet for tablet 57a68056240948c98c4dbbccb5fcc382 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:58:58 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:40742
I20260620 22:58:58.137524 5283 tablet_replica.cc:330] T 57a68056240948c98c4dbbccb5fcc382 P 1e70e1ebf0984593bc40ce95bc7d5500: stopping tablet replica
I20260620 22:58:58.138023 5283 raft_consensus.cc:2229] T 57a68056240948c98c4dbbccb5fcc382 P 1e70e1ebf0984593bc40ce95bc7d5500 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 22:58:58.136545 4728 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43084:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260620 22:58:58.138381 5283 raft_consensus.cc:2258] T 57a68056240948c98c4dbbccb5fcc382 P 1e70e1ebf0984593bc40ce95bc7d5500 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260620 22:58:58.138975 4728 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 22:58:58.140440 5283 ts_tablet_manager.cc:1905] T 57a68056240948c98c4dbbccb5fcc382 P 1e70e1ebf0984593bc40ce95bc7d5500: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:58:58.149688 5146 tablet_service.cc:1467] Processing CreateTablet for tablet c40034bef7c145c09579aab357518a4b (DEFAULT_TABLE table=test-table [id=459dd35ad49c4d6c845f0e3838bdb01d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:58:58.150184 5146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c40034bef7c145c09579aab357518a4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:58:58.150341 5006 tablet_service.cc:1467] Processing CreateTablet for tablet c40034bef7c145c09579aab357518a4b (DEFAULT_TABLE table=test-table [id=459dd35ad49c4d6c845f0e3838bdb01d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:58:58.150503 4867 tablet_service.cc:1467] Processing CreateTablet for tablet c40034bef7c145c09579aab357518a4b (DEFAULT_TABLE table=test-table [id=459dd35ad49c4d6c845f0e3838bdb01d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:58:58.150842 5006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c40034bef7c145c09579aab357518a4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:58:58.151055 4867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c40034bef7c145c09579aab357518a4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:58:58.157027 5236 tablet_bootstrap.cc:492] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b: Bootstrap starting.
I20260620 22:58:58.157400 5235 tablet_bootstrap.cc:492] T c40034bef7c145c09579aab357518a4b P 1e70e1ebf0984593bc40ce95bc7d5500: Bootstrap starting.
I20260620 22:58:58.158704 5236 tablet_bootstrap.cc:654] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b: Neither blocks nor log segments found. Creating new log.
I20260620 22:58:58.159073 5235 tablet_bootstrap.cc:654] T c40034bef7c145c09579aab357518a4b P 1e70e1ebf0984593bc40ce95bc7d5500: Neither blocks nor log segments found. Creating new log.
I20260620 22:58:58.159185 5237 tablet_bootstrap.cc:492] T c40034bef7c145c09579aab357518a4b P f832345cd08e4521bff81fe662f9e938: Bootstrap starting.
I20260620 22:58:58.160722 5237 tablet_bootstrap.cc:654] T c40034bef7c145c09579aab357518a4b P f832345cd08e4521bff81fe662f9e938: Neither blocks nor log segments found. Creating new log.
I20260620 22:58:58.171200 5283 ts_tablet_manager.cc:1918] T 57a68056240948c98c4dbbccb5fcc382 P 1e70e1ebf0984593bc40ce95bc7d5500: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 22:58:58.171443 5283 log.cc:1196] T 57a68056240948c98c4dbbccb5fcc382 P 1e70e1ebf0984593bc40ce95bc7d5500: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-2/wal/wals/57a68056240948c98c4dbbccb5fcc382
I20260620 22:58:58.172101 5283 ts_tablet_manager.cc:1939] T 57a68056240948c98c4dbbccb5fcc382 P 1e70e1ebf0984593bc40ce95bc7d5500: Deleting consensus metadata
I20260620 22:58:58.172199 5235 tablet_bootstrap.cc:492] T c40034bef7c145c09579aab357518a4b P 1e70e1ebf0984593bc40ce95bc7d5500: No bootstrap required, opened a new log
I20260620 22:58:58.172386 5235 ts_tablet_manager.cc:1397] T c40034bef7c145c09579aab357518a4b P 1e70e1ebf0984593bc40ce95bc7d5500: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.003s
I20260620 22:58:58.173367 5235 raft_consensus.cc:348] T c40034bef7c145c09579aab357518a4b P 1e70e1ebf0984593bc40ce95bc7d5500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:58:58.173672 5235 raft_consensus.cc:374] T c40034bef7c145c09579aab357518a4b P 1e70e1ebf0984593bc40ce95bc7d5500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:58:58.173746 5235 raft_consensus.cc:729] T c40034bef7c145c09579aab357518a4b P 1e70e1ebf0984593bc40ce95bc7d5500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e70e1ebf0984593bc40ce95bc7d5500, State: Initialized, Role: FOLLOWER
I20260620 22:58:58.174026 5235 consensus_queue.cc:260] T c40034bef7c145c09579aab357518a4b P 1e70e1ebf0984593bc40ce95bc7d5500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:58:58.174678 5235 ts_tablet_manager.cc:1428] T c40034bef7c145c09579aab357518a4b P 1e70e1ebf0984593bc40ce95bc7d5500: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260620 22:58:58.180639 4717 catalog_manager.cc:4841] TS 1e70e1ebf0984593bc40ce95bc7d5500 (127.2.237.3:40811): tablet 57a68056240948c98c4dbbccb5fcc382 (table test-table [id=d5e317299e5f4b5cb00b55de8be1d969]) successfully deleted
I20260620 22:58:58.182392 5237 tablet_bootstrap.cc:492] T c40034bef7c145c09579aab357518a4b P f832345cd08e4521bff81fe662f9e938: No bootstrap required, opened a new log
I20260620 22:58:58.182600 5237 ts_tablet_manager.cc:1397] T c40034bef7c145c09579aab357518a4b P f832345cd08e4521bff81fe662f9e938: Time spent bootstrapping tablet: real 0.023s user 0.006s sys 0.005s
I20260620 22:58:58.183188 5284 tablet_replica.cc:330] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938: stopping tablet replica
I20260620 22:58:58.183410 5237 raft_consensus.cc:348] T c40034bef7c145c09579aab357518a4b P f832345cd08e4521bff81fe662f9e938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:58:58.183676 5237 raft_consensus.cc:374] T c40034bef7c145c09579aab357518a4b P f832345cd08e4521bff81fe662f9e938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:58:58.183772 5237 raft_consensus.cc:729] T c40034bef7c145c09579aab357518a4b P f832345cd08e4521bff81fe662f9e938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f832345cd08e4521bff81fe662f9e938, State: Initialized, Role: FOLLOWER
I20260620 22:58:58.183907 5284 raft_consensus.cc:2229] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938 [term 1 LEADER]: Raft consensus shutting down.
I20260620 22:58:58.184585 5284 raft_consensus.cc:2258] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:58:58.185135 5282 tablet_replica.cc:330] T 57a68056240948c98c4dbbccb5fcc382 P a0683d16ce0b41fab23ab9d548a9509b: stopping tablet replica
I20260620 22:58:58.184067 5237 consensus_queue.cc:260] T c40034bef7c145c09579aab357518a4b P f832345cd08e4521bff81fe662f9e938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:58:58.185972 5237 ts_tablet_manager.cc:1428] T c40034bef7c145c09579aab357518a4b P f832345cd08e4521bff81fe662f9e938: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260620 22:58:58.186949 5282 raft_consensus.cc:2229] T 57a68056240948c98c4dbbccb5fcc382 P a0683d16ce0b41fab23ab9d548a9509b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 22:58:58.189033 5282 raft_consensus.cc:2258] T 57a68056240948c98c4dbbccb5fcc382 P a0683d16ce0b41fab23ab9d548a9509b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:58:58.188091 5236 tablet_bootstrap.cc:492] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b: No bootstrap required, opened a new log
I20260620 22:58:58.190954 5236 ts_tablet_manager.cc:1397] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.012s
I20260620 22:58:58.191136 5284 ts_tablet_manager.cc:1905] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:58:58.192171 5282 ts_tablet_manager.cc:1905] T 57a68056240948c98c4dbbccb5fcc382 P a0683d16ce0b41fab23ab9d548a9509b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:58:58.192242 5236 raft_consensus.cc:348] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:58:58.194727 5236 raft_consensus.cc:374] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:58:58.194826 5236 raft_consensus.cc:729] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0683d16ce0b41fab23ab9d548a9509b, State: Initialized, Role: FOLLOWER
I20260620 22:58:58.195068 5236 consensus_queue.cc:260] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:58:58.195719 5236 ts_tablet_manager.cc:1428] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260620 22:58:58.200102 5284 ts_tablet_manager.cc:1918] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 22:58:58.200320 5284 log.cc:1196] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-0/wal/wals/57a68056240948c98c4dbbccb5fcc382
I20260620 22:58:58.200882 5284 ts_tablet_manager.cc:1939] T 57a68056240948c98c4dbbccb5fcc382 P f832345cd08e4521bff81fe662f9e938: Deleting consensus metadata
I20260620 22:58:58.202431 5282 ts_tablet_manager.cc:1918] T 57a68056240948c98c4dbbccb5fcc382 P a0683d16ce0b41fab23ab9d548a9509b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 22:58:58.202605 5282 log.cc:1196] T 57a68056240948c98c4dbbccb5fcc382 P a0683d16ce0b41fab23ab9d548a9509b: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-1/wal/wals/57a68056240948c98c4dbbccb5fcc382
I20260620 22:58:58.202625 4716 catalog_manager.cc:4841] TS f832345cd08e4521bff81fe662f9e938 (127.2.237.1:33195): tablet 57a68056240948c98c4dbbccb5fcc382 (table test-table [id=d5e317299e5f4b5cb00b55de8be1d969]) successfully deleted
I20260620 22:58:58.203116 5282 ts_tablet_manager.cc:1939] T 57a68056240948c98c4dbbccb5fcc382 P a0683d16ce0b41fab23ab9d548a9509b: Deleting consensus metadata
I20260620 22:58:58.204682 4716 catalog_manager.cc:4841] TS a0683d16ce0b41fab23ab9d548a9509b (127.2.237.2:33969): tablet 57a68056240948c98c4dbbccb5fcc382 (table test-table [id=d5e317299e5f4b5cb00b55de8be1d969]) successfully deleted
I20260620 22:58:58.406888 5241 raft_consensus.cc:482] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 22:58:58.407168 5241 raft_consensus.cc:504] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:58:58.408932 5241 leader_election.cc:288] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f832345cd08e4521bff81fe662f9e938 (127.2.237.1:33195), 1e70e1ebf0984593bc40ce95bc7d5500 (127.2.237.3:40811)
I20260620 22:58:58.417608 5289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:58.409593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:33195 (local address 127.2.237.2:59371)
0620 22:58:58.410487 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:58.410514 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:58.410710 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:58.411164 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:58.411170 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:58.411196 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:58:58.411529 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:58.411542 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:58.413087 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:58.413093 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:58.415244 (+ 2151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:58.415255 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:58.416558 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:58.416565 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:58.416780 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:58.416830 (+ 50us) client_negotiation.cc:769] Sending connection context
0620 22:58:58.416923 (+ 93us) client_negotiation.cc:240] Negotiation successful
0620 22:58:58.417077 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"mutex_wait_us":1,"spinlock_wait_cycles":11008,"thread_start_us":664,"threads_started":1}
I20260620 22:58:58.417667 5230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:58.409756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:59371 (local address 127.2.237.1:33195)
0620 22:58:58.409996 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:58.410003 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:58.410640 (+ 637us) server_negotiation.cc:407] Connection header received
0620 22:58:58.410915 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:58.410923 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:58.411015 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:58.411117 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:58:58.411766 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:58.412939 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:58.415409 (+ 2470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:58.416379 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:58.416543 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:58.417139 (+ 596us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:58.417213 (+ 74us) server_negotiation.cc:299] Negotiation successful
0620 22:58:58.417349 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260620 22:58:58.418141 5232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:58.409860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:51847 (local address 127.2.237.3:40811)
0620 22:58:58.410037 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:58.410045 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:58.410635 (+ 590us) server_negotiation.cc:407] Connection header received
0620 22:58:58.410945 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:58.410949 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:58.411047 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:58.411161 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:58:58.411934 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:58.412636 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:58.415220 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:58.416477 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:58.416615 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:58.417146 (+ 531us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:58.417727 (+ 581us) server_negotiation.cc:299] Negotiation successful
0620 22:58:58.417892 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260620 22:58:58.418807 4892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c40034bef7c145c09579aab357518a4b" candidate_uuid: "a0683d16ce0b41fab23ab9d548a9509b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f832345cd08e4521bff81fe662f9e938" is_pre_election: true
I20260620 22:58:58.418982 5288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:58.409600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:40811 (local address 127.2.237.2:51847)
0620 22:58:58.410522 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:58.410542 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:58.410721 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:58.411343 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:58.411347 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:58.411371 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:58:58.411779 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:58.411792 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:58.412783 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:58.412792 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:58.415021 (+ 2229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:58.415038 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:58.417321 (+ 2283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:58.417328 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:58.417527 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:58.417566 (+ 39us) client_negotiation.cc:769] Sending connection context
0620 22:58:58.418418 (+ 852us) client_negotiation.cc:240] Negotiation successful
0620 22:58:58.418567 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":727,"mutex_wait_us":262,"thread_start_us":236,"threads_started":1}
I20260620 22:58:58.419101 4892 raft_consensus.cc:2454] T c40034bef7c145c09579aab357518a4b P f832345cd08e4521bff81fe662f9e938 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0683d16ce0b41fab23ab9d548a9509b in term 0.
I20260620 22:58:58.419690 4959 leader_election.cc:302] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0683d16ce0b41fab23ab9d548a9509b, f832345cd08e4521bff81fe662f9e938; no voters:
I20260620 22:58:58.419822 5171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c40034bef7c145c09579aab357518a4b" candidate_uuid: "a0683d16ce0b41fab23ab9d548a9509b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" is_pre_election: true
I20260620 22:58:58.420083 5171 raft_consensus.cc:2454] T c40034bef7c145c09579aab357518a4b P 1e70e1ebf0984593bc40ce95bc7d5500 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0683d16ce0b41fab23ab9d548a9509b in term 0.
I20260620 22:58:58.420149 5241 raft_consensus.cc:2781] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 22:58:58.420270 5241 raft_consensus.cc:482] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 22:58:58.420387 5241 raft_consensus.cc:3037] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:58:58.422818 5241 raft_consensus.cc:504] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:58:58.423576 5241 leader_election.cc:288] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b [CANDIDATE]: Term 1 election: Requested vote from peers f832345cd08e4521bff81fe662f9e938 (127.2.237.1:33195), 1e70e1ebf0984593bc40ce95bc7d5500 (127.2.237.3:40811)
I20260620 22:58:58.424118 4892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c40034bef7c145c09579aab357518a4b" candidate_uuid: "a0683d16ce0b41fab23ab9d548a9509b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f832345cd08e4521bff81fe662f9e938"
I20260620 22:58:58.424165 5171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c40034bef7c145c09579aab357518a4b" candidate_uuid: "a0683d16ce0b41fab23ab9d548a9509b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e70e1ebf0984593bc40ce95bc7d5500"
I20260620 22:58:58.424376 4892 raft_consensus.cc:3037] T c40034bef7c145c09579aab357518a4b P f832345cd08e4521bff81fe662f9e938 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:58:58.424368 5171 raft_consensus.cc:3037] T c40034bef7c145c09579aab357518a4b P 1e70e1ebf0984593bc40ce95bc7d5500 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:58:58.427565 5171 raft_consensus.cc:2454] T c40034bef7c145c09579aab357518a4b P 1e70e1ebf0984593bc40ce95bc7d5500 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0683d16ce0b41fab23ab9d548a9509b in term 1.
I20260620 22:58:58.427999 4892 raft_consensus.cc:2454] T c40034bef7c145c09579aab357518a4b P f832345cd08e4521bff81fe662f9e938 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0683d16ce0b41fab23ab9d548a9509b in term 1.
I20260620 22:58:58.428112 4960 leader_election.cc:302] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e70e1ebf0984593bc40ce95bc7d5500, a0683d16ce0b41fab23ab9d548a9509b; no voters:
I20260620 22:58:58.428629 5241 raft_consensus.cc:2781] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:58:58.429163 5241 raft_consensus.cc:686] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b [term 1 LEADER]: Becoming Leader. State: Replica: a0683d16ce0b41fab23ab9d548a9509b, State: Running, Role: LEADER
I20260620 22:58:58.429587 5241 consensus_queue.cc:237] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:58:58.433590 4728 catalog_manager.cc:5495] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b reported cstate change: term changed from 0 to 1, leader changed from <none> to a0683d16ce0b41fab23ab9d548a9509b (127.2.237.2). New cstate: current_term: 1 leader_uuid: "a0683d16ce0b41fab23ab9d548a9509b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } health_report { overall_health: UNKNOWN } } }
I20260620 22:58:58.476346 5292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:58.470241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36866 (local address 127.2.237.2:33969)
0620 22:58:58.471138 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:58.471142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:58.471160 (+ 18us) server_negotiation.cc:407] Connection header received
0620 22:58:58.471238 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:58.471242 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:58.471309 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:58.471488 (+ 179us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 22:58:58.471918 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:58.472686 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:58.474003 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:58.474427 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:58.474565 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:58.475055 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 22:58:58.475703 (+ 648us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 22:58:58.475876 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:58.475928 (+ 52us) server_negotiation.cc:299] Negotiation successful
0620 22:58:58.476014 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":790,"thread_start_us":682,"threads_started":1}
I20260620 22:58:58.504684 5232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:58.487019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:51827 (local address 127.2.237.3:40811)
0620 22:58:58.487233 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:58.487242 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:58.497705 (+ 10463us) server_negotiation.cc:407] Connection header received
0620 22:58:58.497929 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:58.497935 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:58.498016 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:58.498111 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 22:58:58.498853 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:58.499976 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:58.502065 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:58.503175 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:58.503264 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:58.503905 (+ 641us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 22:58:58.504046 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 22:58:58.504106 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:58.504301 (+ 195us) server_negotiation.cc:299] Negotiation successful
0620 22:58:58.504388 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260620 22:58:58.505379 5295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:58.487003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:40811 (local address 127.2.237.2:51827)
0620 22:58:58.497534 (+ 10531us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:58.497559 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:58.497776 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:58.498262 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:58.498268 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:58.498280 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 22:58:58.498650 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:58.498669 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:58.500148 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:58.500154 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:58.501913 (+ 1759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:58.501927 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:58.503454 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:58.503460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:58.503627 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:58.503698 (+ 71us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 22:58:58.504181 (+ 483us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 22:58:58.504190 (+ 9us) client_negotiation.cc:769] Sending connection context
0620 22:58:58.504868 (+ 678us) client_negotiation.cc:240] Negotiation successful
0620 22:58:58.505010 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10326,"thread_start_us":10095,"threads_started":1}
I20260620 22:58:58.515239 5171 raft_consensus.cc:1261] T c40034bef7c145c09579aab357518a4b P 1e70e1ebf0984593bc40ce95bc7d5500 [term 1 FOLLOWER]: Refusing update from remote peer a0683d16ce0b41fab23ab9d548a9509b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:58:58.515239 4892 raft_consensus.cc:1261] T c40034bef7c145c09579aab357518a4b P f832345cd08e4521bff81fe662f9e938 [term 1 FOLLOWER]: Refusing update from remote peer a0683d16ce0b41fab23ab9d548a9509b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:58:58.515949 5290 consensus_queue.cc:1035] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:58:58.516336 5241 consensus_queue.cc:1035] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:58:58.556620 5279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:58.550208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:33969 (local address 127.2.237.3:53547)
0620 22:58:58.550494 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0620 22:58:58.550514 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 22:58:58.550674 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:58:58.551008 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:58:58.551012 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:58:58.551023 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 22:58:58.551200 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:58.551209 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:58.552567 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:58.552573 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:58.554253 (+ 1680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:58:58.554265 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:58:58.555296 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:58:58.555301 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:58:58.555462 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:58.555555 (+ 93us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 22:58:58.556015 (+ 460us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 22:58:58.556029 (+ 14us) client_negotiation.cc:769] Sending connection context
0620 22:58:58.556106 (+ 77us) client_negotiation.cc:240] Negotiation successful
0620 22:58:58.556281 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147}
I20260620 22:58:58.557374 5292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:58:58.550292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:53547 (local address 127.2.237.2:33969)
0620 22:58:58.550465 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0620 22:58:58.550472 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:58:58.550618 (+ 146us) server_negotiation.cc:407] Connection header received
0620 22:58:58.550817 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:58:58.550822 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:58:58.550897 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:58:58.550994 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 22:58:58.551472 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:58.552424 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:58.554386 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:58:58.555065 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:58:58.555190 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:58:58.555721 (+ 531us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 22:58:58.555861 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 22:58:58.555958 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 22:58:58.556974 (+ 1016us) server_negotiation.cc:299] Negotiation successful
0620 22:58:58.557060 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260620 22:58:58.558488 5265 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260620 22:59:00.162501 4728 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 22:59:00.162825 4728 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 22:59:00.167363 4728 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8ba9bd49121f41eb97acc05b4e83ac70: Sending DeleteTablet for 3 replicas of tablet c40034bef7c145c09579aab357518a4b
I20260620 22:59:00.168177 5007 tablet_service.cc:1514] Processing DeleteTablet for tablet c40034bef7c145c09579aab357518a4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:00 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:36854
I20260620 22:59:00.168255 5146 tablet_service.cc:1514] Processing DeleteTablet for tablet c40034bef7c145c09579aab357518a4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:00 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:56960
I20260620 22:59:00.168249 4868 tablet_service.cc:1514] Processing DeleteTablet for tablet c40034bef7c145c09579aab357518a4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:00 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:40742
I20260620 22:59:00.180512 5325 tablet_replica.cc:330] T c40034bef7c145c09579aab357518a4b P f832345cd08e4521bff81fe662f9e938: stopping tablet replica
I20260620 22:59:00.180528 5323 tablet_replica.cc:330] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b: stopping tablet replica
I20260620 22:59:00.180754 5325 raft_consensus.cc:2229] T c40034bef7c145c09579aab357518a4b P f832345cd08e4521bff81fe662f9e938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 22:59:00.180771 5323 raft_consensus.cc:2229] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b [term 1 LEADER]: Raft consensus shutting down.
I20260620 22:59:00.180995 5325 raft_consensus.cc:2258] T c40034bef7c145c09579aab357518a4b P f832345cd08e4521bff81fe662f9e938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:00.182900 5325 ts_tablet_manager.cc:1905] T c40034bef7c145c09579aab357518a4b P f832345cd08e4521bff81fe662f9e938: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:00.182965 5324 tablet_replica.cc:330] T c40034bef7c145c09579aab357518a4b P 1e70e1ebf0984593bc40ce95bc7d5500: stopping tablet replica
I20260620 22:59:00.183130 5324 raft_consensus.cc:2229] T c40034bef7c145c09579aab357518a4b P 1e70e1ebf0984593bc40ce95bc7d5500 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 22:59:00.183229 5323 raft_consensus.cc:2258] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:00.183460 5324 raft_consensus.cc:2258] T c40034bef7c145c09579aab357518a4b P 1e70e1ebf0984593bc40ce95bc7d5500 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:00.184299 5323 ts_tablet_manager.cc:1905] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:00.184357 5324 ts_tablet_manager.cc:1905] T c40034bef7c145c09579aab357518a4b P 1e70e1ebf0984593bc40ce95bc7d5500: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:00.189921 5325 ts_tablet_manager.cc:1918] T c40034bef7c145c09579aab357518a4b P f832345cd08e4521bff81fe662f9e938: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 22:59:00.190109 5325 log.cc:1196] T c40034bef7c145c09579aab357518a4b P f832345cd08e4521bff81fe662f9e938: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-0/wal/wals/c40034bef7c145c09579aab357518a4b
I20260620 22:59:00.191078 5325 ts_tablet_manager.cc:1939] T c40034bef7c145c09579aab357518a4b P f832345cd08e4521bff81fe662f9e938: Deleting consensus metadata
I20260620 22:59:00.191469 5324 ts_tablet_manager.cc:1918] T c40034bef7c145c09579aab357518a4b P 1e70e1ebf0984593bc40ce95bc7d5500: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 22:59:00.191529 5323 ts_tablet_manager.cc:1918] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 22:59:00.191673 5324 log.cc:1196] T c40034bef7c145c09579aab357518a4b P 1e70e1ebf0984593bc40ce95bc7d5500: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-2/wal/wals/c40034bef7c145c09579aab357518a4b
I20260620 22:59:00.191735 5323 log.cc:1196] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-1/wal/wals/c40034bef7c145c09579aab357518a4b
I20260620 22:59:00.192178 5324 ts_tablet_manager.cc:1939] T c40034bef7c145c09579aab357518a4b P 1e70e1ebf0984593bc40ce95bc7d5500: Deleting consensus metadata
I20260620 22:59:00.192243 5323 ts_tablet_manager.cc:1939] T c40034bef7c145c09579aab357518a4b P a0683d16ce0b41fab23ab9d548a9509b: Deleting consensus metadata
I20260620 22:59:00.192358 4716 catalog_manager.cc:4841] TS f832345cd08e4521bff81fe662f9e938 (127.2.237.1:33195): tablet c40034bef7c145c09579aab357518a4b (table test-table [id=459dd35ad49c4d6c845f0e3838bdb01d]) successfully deleted
I20260620 22:59:00.193275 4717 catalog_manager.cc:4841] TS 1e70e1ebf0984593bc40ce95bc7d5500 (127.2.237.3:40811): tablet c40034bef7c145c09579aab357518a4b (table test-table [id=459dd35ad49c4d6c845f0e3838bdb01d]) successfully deleted
I20260620 22:59:00.193452 4716 catalog_manager.cc:4841] TS a0683d16ce0b41fab23ab9d548a9509b (127.2.237.2:33969): tablet c40034bef7c145c09579aab357518a4b (table test-table [id=459dd35ad49c4d6c845f0e3838bdb01d]) successfully deleted
Jun 20 22:59:00 dist-test-slave-f63b krb5kdc[4686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996340, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 22:59:00 dist-test-slave-f63b krb5kdc[4686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996340, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:59:00.226152 5333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:00.213192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43094 (local address 127.2.237.62:37999)
0620 22:59:00.213710 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:00.213714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:00.213784 (+ 70us) server_negotiation.cc:407] Connection header received
0620 22:59:00.213942 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:00.213946 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:00.214009 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:00.214106 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:00.215453 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:00.216175 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:00.217396 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:00.217769 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:00.217856 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:00.221566 (+ 3710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:00.221571 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:00.221574 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:00.221604 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:00.224118 (+ 2514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:00.224673 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:00.224676 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:00.224678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:00.224784 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:00.225079 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:00.225082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:00.225084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:00.225334 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:00.225505 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:00.225732 (+ 227us) server_negotiation.cc:299] Negotiation successful
0620 22:59:00.225887 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":274,"threads_started":1}
I20260620 22:59:00.229527 4728 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43094:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 22:59:00.230082 4728 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 22:59:00.236227 4868 tablet_service.cc:1467] Processing CreateTablet for tablet 37ce0ccbb2b44ef58460c8b004c38508 (DEFAULT_TABLE table=test-table [id=1890a26797ae494e999cb64f1b8b571f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:59:00.236289 5007 tablet_service.cc:1467] Processing CreateTablet for tablet 37ce0ccbb2b44ef58460c8b004c38508 (DEFAULT_TABLE table=test-table [id=1890a26797ae494e999cb64f1b8b571f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:59:00.236528 5146 tablet_service.cc:1467] Processing CreateTablet for tablet 37ce0ccbb2b44ef58460c8b004c38508 (DEFAULT_TABLE table=test-table [id=1890a26797ae494e999cb64f1b8b571f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:59:00.236676 4868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37ce0ccbb2b44ef58460c8b004c38508. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:00.236675 5007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37ce0ccbb2b44ef58460c8b004c38508. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:00.236881 5146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37ce0ccbb2b44ef58460c8b004c38508. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:00.242663 5334 tablet_bootstrap.cc:492] T 37ce0ccbb2b44ef58460c8b004c38508 P 1e70e1ebf0984593bc40ce95bc7d5500: Bootstrap starting.
I20260620 22:59:00.243813 5336 tablet_bootstrap.cc:492] T 37ce0ccbb2b44ef58460c8b004c38508 P f832345cd08e4521bff81fe662f9e938: Bootstrap starting.
I20260620 22:59:00.244309 5334 tablet_bootstrap.cc:654] T 37ce0ccbb2b44ef58460c8b004c38508 P 1e70e1ebf0984593bc40ce95bc7d5500: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:00.244349 5335 tablet_bootstrap.cc:492] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b: Bootstrap starting.
I20260620 22:59:00.246044 5335 tablet_bootstrap.cc:654] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:00.246377 5334 tablet_bootstrap.cc:492] T 37ce0ccbb2b44ef58460c8b004c38508 P 1e70e1ebf0984593bc40ce95bc7d5500: No bootstrap required, opened a new log
I20260620 22:59:00.246876 5334 ts_tablet_manager.cc:1397] T 37ce0ccbb2b44ef58460c8b004c38508 P 1e70e1ebf0984593bc40ce95bc7d5500: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260620 22:59:00.247538 5336 tablet_bootstrap.cc:654] T 37ce0ccbb2b44ef58460c8b004c38508 P f832345cd08e4521bff81fe662f9e938: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:00.247611 5334 raft_consensus.cc:348] T 37ce0ccbb2b44ef58460c8b004c38508 P 1e70e1ebf0984593bc40ce95bc7d5500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:59:00.247864 5334 raft_consensus.cc:374] T 37ce0ccbb2b44ef58460c8b004c38508 P 1e70e1ebf0984593bc40ce95bc7d5500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:00.248013 5334 raft_consensus.cc:729] T 37ce0ccbb2b44ef58460c8b004c38508 P 1e70e1ebf0984593bc40ce95bc7d5500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e70e1ebf0984593bc40ce95bc7d5500, State: Initialized, Role: FOLLOWER
I20260620 22:59:00.248272 5334 consensus_queue.cc:260] T 37ce0ccbb2b44ef58460c8b004c38508 P 1e70e1ebf0984593bc40ce95bc7d5500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:59:00.248858 5335 tablet_bootstrap.cc:492] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b: No bootstrap required, opened a new log
I20260620 22:59:00.249034 5335 ts_tablet_manager.cc:1397] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260620 22:59:00.249739 5336 tablet_bootstrap.cc:492] T 37ce0ccbb2b44ef58460c8b004c38508 P f832345cd08e4521bff81fe662f9e938: No bootstrap required, opened a new log
I20260620 22:59:00.249864 5334 ts_tablet_manager.cc:1428] T 37ce0ccbb2b44ef58460c8b004c38508 P 1e70e1ebf0984593bc40ce95bc7d5500: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260620 22:59:00.249945 5336 ts_tablet_manager.cc:1397] T 37ce0ccbb2b44ef58460c8b004c38508 P f832345cd08e4521bff81fe662f9e938: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260620 22:59:00.249953 5335 raft_consensus.cc:348] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:59:00.250180 5335 raft_consensus.cc:374] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:00.250300 5335 raft_consensus.cc:729] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0683d16ce0b41fab23ab9d548a9509b, State: Initialized, Role: FOLLOWER
I20260620 22:59:00.250538 5335 consensus_queue.cc:260] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:59:00.250818 5336 raft_consensus.cc:348] T 37ce0ccbb2b44ef58460c8b004c38508 P f832345cd08e4521bff81fe662f9e938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:59:00.251127 5335 ts_tablet_manager.cc:1428] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260620 22:59:00.251120 5336 raft_consensus.cc:374] T 37ce0ccbb2b44ef58460c8b004c38508 P f832345cd08e4521bff81fe662f9e938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:00.251266 5336 raft_consensus.cc:729] T 37ce0ccbb2b44ef58460c8b004c38508 P f832345cd08e4521bff81fe662f9e938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f832345cd08e4521bff81fe662f9e938, State: Initialized, Role: FOLLOWER
I20260620 22:59:00.251529 5336 consensus_queue.cc:260] T 37ce0ccbb2b44ef58460c8b004c38508 P f832345cd08e4521bff81fe662f9e938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:59:00.252480 5336 ts_tablet_manager.cc:1428] T 37ce0ccbb2b44ef58460c8b004c38508 P f832345cd08e4521bff81fe662f9e938: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 22:59:00.611171 5318 raft_consensus.cc:482] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 22:59:00.611388 5318 raft_consensus.cc:504] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:59:00.612221 5318 leader_election.cc:288] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f832345cd08e4521bff81fe662f9e938 (127.2.237.1:33195), 1e70e1ebf0984593bc40ce95bc7d5500 (127.2.237.3:40811)
I20260620 22:59:00.612775 4892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37ce0ccbb2b44ef58460c8b004c38508" candidate_uuid: "a0683d16ce0b41fab23ab9d548a9509b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f832345cd08e4521bff81fe662f9e938" is_pre_election: true
I20260620 22:59:00.612794 5171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37ce0ccbb2b44ef58460c8b004c38508" candidate_uuid: "a0683d16ce0b41fab23ab9d548a9509b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" is_pre_election: true
I20260620 22:59:00.613006 4892 raft_consensus.cc:2454] T 37ce0ccbb2b44ef58460c8b004c38508 P f832345cd08e4521bff81fe662f9e938 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0683d16ce0b41fab23ab9d548a9509b in term 0.
I20260620 22:59:00.613047 5171 raft_consensus.cc:2454] T 37ce0ccbb2b44ef58460c8b004c38508 P 1e70e1ebf0984593bc40ce95bc7d5500 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0683d16ce0b41fab23ab9d548a9509b in term 0.
I20260620 22:59:00.613509 4959 leader_election.cc:302] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0683d16ce0b41fab23ab9d548a9509b, f832345cd08e4521bff81fe662f9e938; no voters:
I20260620 22:59:00.613809 5318 raft_consensus.cc:2781] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 22:59:00.613907 5318 raft_consensus.cc:482] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 22:59:00.614017 5318 raft_consensus.cc:3037] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:00.616076 5318 raft_consensus.cc:504] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:59:00.616616 5318 leader_election.cc:288] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b [CANDIDATE]: Term 1 election: Requested vote from peers f832345cd08e4521bff81fe662f9e938 (127.2.237.1:33195), 1e70e1ebf0984593bc40ce95bc7d5500 (127.2.237.3:40811)
I20260620 22:59:00.617070 5171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37ce0ccbb2b44ef58460c8b004c38508" candidate_uuid: "a0683d16ce0b41fab23ab9d548a9509b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e70e1ebf0984593bc40ce95bc7d5500"
I20260620 22:59:00.617117 4892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37ce0ccbb2b44ef58460c8b004c38508" candidate_uuid: "a0683d16ce0b41fab23ab9d548a9509b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f832345cd08e4521bff81fe662f9e938"
I20260620 22:59:00.617264 5171 raft_consensus.cc:3037] T 37ce0ccbb2b44ef58460c8b004c38508 P 1e70e1ebf0984593bc40ce95bc7d5500 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:00.617358 4892 raft_consensus.cc:3037] T 37ce0ccbb2b44ef58460c8b004c38508 P f832345cd08e4521bff81fe662f9e938 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:00.619345 5171 raft_consensus.cc:2454] T 37ce0ccbb2b44ef58460c8b004c38508 P 1e70e1ebf0984593bc40ce95bc7d5500 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0683d16ce0b41fab23ab9d548a9509b in term 1.
I20260620 22:59:00.619514 4892 raft_consensus.cc:2454] T 37ce0ccbb2b44ef58460c8b004c38508 P f832345cd08e4521bff81fe662f9e938 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0683d16ce0b41fab23ab9d548a9509b in term 1.
I20260620 22:59:00.619732 4960 leader_election.cc:302] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e70e1ebf0984593bc40ce95bc7d5500, a0683d16ce0b41fab23ab9d548a9509b; no voters:
I20260620 22:59:00.620002 5318 raft_consensus.cc:2781] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:59:00.620142 5318 raft_consensus.cc:686] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b [term 1 LEADER]: Becoming Leader. State: Replica: a0683d16ce0b41fab23ab9d548a9509b, State: Running, Role: LEADER
I20260620 22:59:00.620438 5318 consensus_queue.cc:237] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:59:00.623116 4728 catalog_manager.cc:5495] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b reported cstate change: term changed from 0 to 1, leader changed from <none> to a0683d16ce0b41fab23ab9d548a9509b (127.2.237.2). New cstate: current_term: 1 leader_uuid: "a0683d16ce0b41fab23ab9d548a9509b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } health_report { overall_health: UNKNOWN } } }
I20260620 22:59:00.711328 5351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:00.704421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36876 (local address 127.2.237.2:33969)
0620 22:59:00.704946 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:00.704953 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:00.704971 (+ 18us) server_negotiation.cc:407] Connection header received
0620 22:59:00.705037 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:00.705042 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:00.705118 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:00.705246 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 22:59:00.705617 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:00.707291 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:00.709344 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:00.710002 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:00.710126 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:00.710765 (+ 639us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 22:59:00.710861 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 22:59:00.710929 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:00.711039 (+ 110us) server_negotiation.cc:299] Negotiation successful
0620 22:59:00.711094 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":236,"threads_started":1}
I20260620 22:59:00.719594 5171 raft_consensus.cc:1261] T 37ce0ccbb2b44ef58460c8b004c38508 P 1e70e1ebf0984593bc40ce95bc7d5500 [term 1 FOLLOWER]: Refusing update from remote peer a0683d16ce0b41fab23ab9d548a9509b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:59:00.720322 4892 raft_consensus.cc:1261] T 37ce0ccbb2b44ef58460c8b004c38508 P f832345cd08e4521bff81fe662f9e938 [term 1 FOLLOWER]: Refusing update from remote peer a0683d16ce0b41fab23ab9d548a9509b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:59:00.720214 5300 consensus_queue.cc:1035] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:59:00.721016 5300 consensus_queue.cc:1035] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:59:00.767733 4728 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 22:59:00.768011 4728 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 22:59:00.770973 4728 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8ba9bd49121f41eb97acc05b4e83ac70: Sending DeleteTablet for 3 replicas of tablet 37ce0ccbb2b44ef58460c8b004c38508
I20260620 22:59:00.771808 4868 tablet_service.cc:1514] Processing DeleteTablet for tablet 37ce0ccbb2b44ef58460c8b004c38508 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:00 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:40742
I20260620 22:59:00.771811 5146 tablet_service.cc:1514] Processing DeleteTablet for tablet 37ce0ccbb2b44ef58460c8b004c38508 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:00 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:56960
I20260620 22:59:00.771836 5007 tablet_service.cc:1514] Processing DeleteTablet for tablet 37ce0ccbb2b44ef58460c8b004c38508 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:00 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:36854
I20260620 22:59:00.772960 4728 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43094:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 22:59:00.773993 4728 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 22:59:00.776701 5360 tablet_replica.cc:330] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b: stopping tablet replica
I20260620 22:59:00.776939 5360 raft_consensus.cc:2229] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b [term 1 LEADER]: Raft consensus shutting down.
I20260620 22:59:00.777009 5358 tablet_replica.cc:330] T 37ce0ccbb2b44ef58460c8b004c38508 P f832345cd08e4521bff81fe662f9e938: stopping tablet replica
I20260620 22:59:00.777328 5358 raft_consensus.cc:2229] T 37ce0ccbb2b44ef58460c8b004c38508 P f832345cd08e4521bff81fe662f9e938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 22:59:00.777359 5360 raft_consensus.cc:2258] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:00.777581 5358 raft_consensus.cc:2258] T 37ce0ccbb2b44ef58460c8b004c38508 P f832345cd08e4521bff81fe662f9e938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:00.778373 5360 ts_tablet_manager.cc:1905] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:00.778626 5358 ts_tablet_manager.cc:1905] T 37ce0ccbb2b44ef58460c8b004c38508 P f832345cd08e4521bff81fe662f9e938: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:00.782451 5145 tablet_service.cc:1467] Processing CreateTablet for tablet 2f5bf573c8934fd3b23aa9f3761700c4 (DEFAULT_TABLE table=test-table [id=e2623abdf54d41d382652c73fc87f32c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:59:00.782997 5145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f5bf573c8934fd3b23aa9f3761700c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:00.785413 5360 ts_tablet_manager.cc:1918] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 22:59:00.785449 5358 ts_tablet_manager.cc:1918] T 37ce0ccbb2b44ef58460c8b004c38508 P f832345cd08e4521bff81fe662f9e938: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 22:59:00.785531 5360 log.cc:1196] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-1/wal/wals/37ce0ccbb2b44ef58460c8b004c38508
I20260620 22:59:00.785552 5358 log.cc:1196] T 37ce0ccbb2b44ef58460c8b004c38508 P f832345cd08e4521bff81fe662f9e938: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-0/wal/wals/37ce0ccbb2b44ef58460c8b004c38508
I20260620 22:59:00.785969 5360 ts_tablet_manager.cc:1939] T 37ce0ccbb2b44ef58460c8b004c38508 P a0683d16ce0b41fab23ab9d548a9509b: Deleting consensus metadata
I20260620 22:59:00.785974 5358 ts_tablet_manager.cc:1939] T 37ce0ccbb2b44ef58460c8b004c38508 P f832345cd08e4521bff81fe662f9e938: Deleting consensus metadata
I20260620 22:59:00.787271 5359 tablet_replica.cc:330] T 37ce0ccbb2b44ef58460c8b004c38508 P 1e70e1ebf0984593bc40ce95bc7d5500: stopping tablet replica
I20260620 22:59:00.787433 4716 catalog_manager.cc:4841] TS f832345cd08e4521bff81fe662f9e938 (127.2.237.1:33195): tablet 37ce0ccbb2b44ef58460c8b004c38508 (table test-table [id=1890a26797ae494e999cb64f1b8b571f]) successfully deleted
I20260620 22:59:00.787500 5359 raft_consensus.cc:2229] T 37ce0ccbb2b44ef58460c8b004c38508 P 1e70e1ebf0984593bc40ce95bc7d5500 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 22:59:00.787644 4716 catalog_manager.cc:4841] TS a0683d16ce0b41fab23ab9d548a9509b (127.2.237.2:33969): tablet 37ce0ccbb2b44ef58460c8b004c38508 (table test-table [id=1890a26797ae494e999cb64f1b8b571f]) successfully deleted
I20260620 22:59:00.787740 4868 tablet_service.cc:1467] Processing CreateTablet for tablet 2f5bf573c8934fd3b23aa9f3761700c4 (DEFAULT_TABLE table=test-table [id=e2623abdf54d41d382652c73fc87f32c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:59:00.787837 5359 raft_consensus.cc:2258] T 37ce0ccbb2b44ef58460c8b004c38508 P 1e70e1ebf0984593bc40ce95bc7d5500 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:00.788136 4868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f5bf573c8934fd3b23aa9f3761700c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:00.788767 5007 tablet_service.cc:1467] Processing CreateTablet for tablet 2f5bf573c8934fd3b23aa9f3761700c4 (DEFAULT_TABLE table=test-table [id=e2623abdf54d41d382652c73fc87f32c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:59:00.788911 5359 ts_tablet_manager.cc:1905] T 37ce0ccbb2b44ef58460c8b004c38508 P 1e70e1ebf0984593bc40ce95bc7d5500: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:00.789137 5007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f5bf573c8934fd3b23aa9f3761700c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:00.796206 5359 ts_tablet_manager.cc:1918] T 37ce0ccbb2b44ef58460c8b004c38508 P 1e70e1ebf0984593bc40ce95bc7d5500: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 22:59:00.796474 5359 log.cc:1196] T 37ce0ccbb2b44ef58460c8b004c38508 P 1e70e1ebf0984593bc40ce95bc7d5500: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-2/wal/wals/37ce0ccbb2b44ef58460c8b004c38508
I20260620 22:59:00.797127 5359 ts_tablet_manager.cc:1939] T 37ce0ccbb2b44ef58460c8b004c38508 P 1e70e1ebf0984593bc40ce95bc7d5500: Deleting consensus metadata
I20260620 22:59:00.798866 5362 tablet_bootstrap.cc:492] T 2f5bf573c8934fd3b23aa9f3761700c4 P f832345cd08e4521bff81fe662f9e938: Bootstrap starting.
I20260620 22:59:00.799058 4717 catalog_manager.cc:4841] TS 1e70e1ebf0984593bc40ce95bc7d5500 (127.2.237.3:40811): tablet 37ce0ccbb2b44ef58460c8b004c38508 (table test-table [id=1890a26797ae494e999cb64f1b8b571f]) successfully deleted
I20260620 22:59:00.800500 5361 tablet_bootstrap.cc:492] T 2f5bf573c8934fd3b23aa9f3761700c4 P 1e70e1ebf0984593bc40ce95bc7d5500: Bootstrap starting.
I20260620 22:59:00.800542 5362 tablet_bootstrap.cc:654] T 2f5bf573c8934fd3b23aa9f3761700c4 P f832345cd08e4521bff81fe662f9e938: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:00.802139 5363 tablet_bootstrap.cc:492] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b: Bootstrap starting.
I20260620 22:59:00.802671 5362 tablet_bootstrap.cc:492] T 2f5bf573c8934fd3b23aa9f3761700c4 P f832345cd08e4521bff81fe662f9e938: No bootstrap required, opened a new log
I20260620 22:59:00.802811 5362 ts_tablet_manager.cc:1397] T 2f5bf573c8934fd3b23aa9f3761700c4 P f832345cd08e4521bff81fe662f9e938: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260620 22:59:00.802871 5361 tablet_bootstrap.cc:654] T 2f5bf573c8934fd3b23aa9f3761700c4 P 1e70e1ebf0984593bc40ce95bc7d5500: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:00.803541 5362 raft_consensus.cc:348] T 2f5bf573c8934fd3b23aa9f3761700c4 P f832345cd08e4521bff81fe662f9e938 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:59:00.803835 5362 raft_consensus.cc:374] T 2f5bf573c8934fd3b23aa9f3761700c4 P f832345cd08e4521bff81fe662f9e938 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:00.803867 5363 tablet_bootstrap.cc:654] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:00.803932 5362 raft_consensus.cc:729] T 2f5bf573c8934fd3b23aa9f3761700c4 P f832345cd08e4521bff81fe662f9e938 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f832345cd08e4521bff81fe662f9e938, State: Initialized, Role: FOLLOWER
I20260620 22:59:00.804574 5362 consensus_queue.cc:260] T 2f5bf573c8934fd3b23aa9f3761700c4 P f832345cd08e4521bff81fe662f9e938 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:59:00.805016 5361 tablet_bootstrap.cc:492] T 2f5bf573c8934fd3b23aa9f3761700c4 P 1e70e1ebf0984593bc40ce95bc7d5500: No bootstrap required, opened a new log
I20260620 22:59:00.805176 5361 ts_tablet_manager.cc:1397] T 2f5bf573c8934fd3b23aa9f3761700c4 P 1e70e1ebf0984593bc40ce95bc7d5500: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260620 22:59:00.805981 5362 ts_tablet_manager.cc:1428] T 2f5bf573c8934fd3b23aa9f3761700c4 P f832345cd08e4521bff81fe662f9e938: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260620 22:59:00.806108 5361 raft_consensus.cc:348] T 2f5bf573c8934fd3b23aa9f3761700c4 P 1e70e1ebf0984593bc40ce95bc7d5500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:59:00.806422 5361 raft_consensus.cc:374] T 2f5bf573c8934fd3b23aa9f3761700c4 P 1e70e1ebf0984593bc40ce95bc7d5500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:00.806560 5361 raft_consensus.cc:729] T 2f5bf573c8934fd3b23aa9f3761700c4 P 1e70e1ebf0984593bc40ce95bc7d5500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e70e1ebf0984593bc40ce95bc7d5500, State: Initialized, Role: FOLLOWER
I20260620 22:59:00.806893 5361 consensus_queue.cc:260] T 2f5bf573c8934fd3b23aa9f3761700c4 P 1e70e1ebf0984593bc40ce95bc7d5500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:59:00.807561 5361 ts_tablet_manager.cc:1428] T 2f5bf573c8934fd3b23aa9f3761700c4 P 1e70e1ebf0984593bc40ce95bc7d5500: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260620 22:59:00.807606 5363 tablet_bootstrap.cc:492] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b: No bootstrap required, opened a new log
I20260620 22:59:00.807832 5363 ts_tablet_manager.cc:1397] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260620 22:59:00.808678 5363 raft_consensus.cc:348] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:59:00.808983 5363 raft_consensus.cc:374] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:00.809108 5363 raft_consensus.cc:729] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0683d16ce0b41fab23ab9d548a9509b, State: Initialized, Role: FOLLOWER
I20260620 22:59:00.809392 5363 consensus_queue.cc:260] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:59:00.809960 5363 ts_tablet_manager.cc:1428] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260620 22:59:00.885892 5355 raft_consensus.cc:482] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 22:59:00.886142 5355 raft_consensus.cc:504] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:59:00.887110 5355 leader_election.cc:288] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f832345cd08e4521bff81fe662f9e938 (127.2.237.1:33195), 1e70e1ebf0984593bc40ce95bc7d5500 (127.2.237.3:40811)
I20260620 22:59:00.887621 4892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f5bf573c8934fd3b23aa9f3761700c4" candidate_uuid: "a0683d16ce0b41fab23ab9d548a9509b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f832345cd08e4521bff81fe662f9e938" is_pre_election: true
I20260620 22:59:00.887677 5171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f5bf573c8934fd3b23aa9f3761700c4" candidate_uuid: "a0683d16ce0b41fab23ab9d548a9509b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" is_pre_election: true
I20260620 22:59:00.887882 4892 raft_consensus.cc:2454] T 2f5bf573c8934fd3b23aa9f3761700c4 P f832345cd08e4521bff81fe662f9e938 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0683d16ce0b41fab23ab9d548a9509b in term 0.
I20260620 22:59:00.887985 5171 raft_consensus.cc:2454] T 2f5bf573c8934fd3b23aa9f3761700c4 P 1e70e1ebf0984593bc40ce95bc7d5500 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0683d16ce0b41fab23ab9d548a9509b in term 0.
I20260620 22:59:00.888407 4959 leader_election.cc:302] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0683d16ce0b41fab23ab9d548a9509b, f832345cd08e4521bff81fe662f9e938; no voters:
I20260620 22:59:00.888710 5355 raft_consensus.cc:2781] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 22:59:00.888864 5355 raft_consensus.cc:482] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 22:59:00.888964 5355 raft_consensus.cc:3037] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:00.892661 5355 raft_consensus.cc:504] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:59:00.893421 5355 leader_election.cc:288] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b [CANDIDATE]: Term 1 election: Requested vote from peers f832345cd08e4521bff81fe662f9e938 (127.2.237.1:33195), 1e70e1ebf0984593bc40ce95bc7d5500 (127.2.237.3:40811)
I20260620 22:59:00.893973 5171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f5bf573c8934fd3b23aa9f3761700c4" candidate_uuid: "a0683d16ce0b41fab23ab9d548a9509b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e70e1ebf0984593bc40ce95bc7d5500"
I20260620 22:59:00.894150 4892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f5bf573c8934fd3b23aa9f3761700c4" candidate_uuid: "a0683d16ce0b41fab23ab9d548a9509b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f832345cd08e4521bff81fe662f9e938"
I20260620 22:59:00.894248 5171 raft_consensus.cc:3037] T 2f5bf573c8934fd3b23aa9f3761700c4 P 1e70e1ebf0984593bc40ce95bc7d5500 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:00.894362 4892 raft_consensus.cc:3037] T 2f5bf573c8934fd3b23aa9f3761700c4 P f832345cd08e4521bff81fe662f9e938 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:00.896878 4892 raft_consensus.cc:2454] T 2f5bf573c8934fd3b23aa9f3761700c4 P f832345cd08e4521bff81fe662f9e938 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0683d16ce0b41fab23ab9d548a9509b in term 1.
I20260620 22:59:00.897353 4959 leader_election.cc:302] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0683d16ce0b41fab23ab9d548a9509b, f832345cd08e4521bff81fe662f9e938; no voters:
I20260620 22:59:00.897663 5355 raft_consensus.cc:2781] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:59:00.897832 5355 raft_consensus.cc:686] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b [term 1 LEADER]: Becoming Leader. State: Replica: a0683d16ce0b41fab23ab9d548a9509b, State: Running, Role: LEADER
I20260620 22:59:00.897835 5171 raft_consensus.cc:2454] T 2f5bf573c8934fd3b23aa9f3761700c4 P 1e70e1ebf0984593bc40ce95bc7d5500 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0683d16ce0b41fab23ab9d548a9509b in term 1.
I20260620 22:59:00.898151 5355 consensus_queue.cc:237] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } }
I20260620 22:59:00.901117 4728 catalog_manager.cc:5495] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b reported cstate change: term changed from 0 to 1, leader changed from <none> to a0683d16ce0b41fab23ab9d548a9509b (127.2.237.2). New cstate: current_term: 1 leader_uuid: "a0683d16ce0b41fab23ab9d548a9509b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0683d16ce0b41fab23ab9d548a9509b" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33969 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 } health_report { overall_health: UNKNOWN } } }
I20260620 22:59:00.943233 4892 raft_consensus.cc:1261] T 2f5bf573c8934fd3b23aa9f3761700c4 P f832345cd08e4521bff81fe662f9e938 [term 1 FOLLOWER]: Refusing update from remote peer a0683d16ce0b41fab23ab9d548a9509b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:59:00.943441 5171 raft_consensus.cc:1261] T 2f5bf573c8934fd3b23aa9f3761700c4 P 1e70e1ebf0984593bc40ce95bc7d5500 [term 1 FOLLOWER]: Refusing update from remote peer a0683d16ce0b41fab23ab9d548a9509b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:59:00.943892 5355 consensus_queue.cc:1035] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f832345cd08e4521bff81fe662f9e938" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:59:00.944190 5356 consensus_queue.cc:1035] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e70e1ebf0984593bc40ce95bc7d5500" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 22:59:00.969905 5344 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260620 22:59:02.567034 4730 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 22:59:02.567293 4730 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43094:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 22:59:02.571113 4730 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8ba9bd49121f41eb97acc05b4e83ac70: Sending DeleteTablet for 3 replicas of tablet 2f5bf573c8934fd3b23aa9f3761700c4
I20260620 22:59:02.571910 4868 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f5bf573c8934fd3b23aa9f3761700c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:02 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:40742
I20260620 22:59:02.571955 5007 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f5bf573c8934fd3b23aa9f3761700c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:02 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:36854
I20260620 22:59:02.572073 5145 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f5bf573c8934fd3b23aa9f3761700c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:02 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:56960
I20260620 22:59:02.572911 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 4802
I20260620 22:59:02.584378 5392 tablet_replica.cc:330] T 2f5bf573c8934fd3b23aa9f3761700c4 P f832345cd08e4521bff81fe662f9e938: stopping tablet replica
I20260620 22:59:02.584832 5392 raft_consensus.cc:2229] T 2f5bf573c8934fd3b23aa9f3761700c4 P f832345cd08e4521bff81fe662f9e938 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 22:59:02.585283 5392 raft_consensus.cc:2258] T 2f5bf573c8934fd3b23aa9f3761700c4 P f832345cd08e4521bff81fe662f9e938 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:02.586730 5392 ts_tablet_manager.cc:1905] T 2f5bf573c8934fd3b23aa9f3761700c4 P f832345cd08e4521bff81fe662f9e938: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:02.587385 5394 tablet_replica.cc:330] T 2f5bf573c8934fd3b23aa9f3761700c4 P 1e70e1ebf0984593bc40ce95bc7d5500: stopping tablet replica
I20260620 22:59:02.587790 5394 raft_consensus.cc:2229] T 2f5bf573c8934fd3b23aa9f3761700c4 P 1e70e1ebf0984593bc40ce95bc7d5500 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 22:59:02.588151 5394 raft_consensus.cc:2258] T 2f5bf573c8934fd3b23aa9f3761700c4 P 1e70e1ebf0984593bc40ce95bc7d5500 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:02.588750 5393 tablet_replica.cc:330] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b: stopping tablet replica
I20260620 22:59:02.589033 5393 raft_consensus.cc:2229] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b [term 1 LEADER]: Raft consensus shutting down.
I20260620 22:59:02.589437 5393 raft_consensus.cc:2258] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:02.589582 5394 ts_tablet_manager.cc:1905] T 2f5bf573c8934fd3b23aa9f3761700c4 P 1e70e1ebf0984593bc40ce95bc7d5500: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:02.590934 5393 ts_tablet_manager.cc:1905] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:02.595438 5392 ts_tablet_manager.cc:1918] T 2f5bf573c8934fd3b23aa9f3761700c4 P f832345cd08e4521bff81fe662f9e938: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 22:59:02.595674 5392 log.cc:1196] T 2f5bf573c8934fd3b23aa9f3761700c4 P f832345cd08e4521bff81fe662f9e938: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-0/wal/wals/2f5bf573c8934fd3b23aa9f3761700c4
I20260620 22:59:02.596237 5392 ts_tablet_manager.cc:1939] T 2f5bf573c8934fd3b23aa9f3761700c4 P f832345cd08e4521bff81fe662f9e938: Deleting consensus metadata
I20260620 22:59:02.597481 4716 catalog_manager.cc:4841] TS f832345cd08e4521bff81fe662f9e938 (127.2.237.1:33195): tablet 2f5bf573c8934fd3b23aa9f3761700c4 (table test-table [id=e2623abdf54d41d382652c73fc87f32c]) successfully deleted
I20260620 22:59:02.597360 5394 ts_tablet_manager.cc:1918] T 2f5bf573c8934fd3b23aa9f3761700c4 P 1e70e1ebf0984593bc40ce95bc7d5500: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 22:59:02.597918 5394 log.cc:1196] T 2f5bf573c8934fd3b23aa9f3761700c4 P 1e70e1ebf0984593bc40ce95bc7d5500: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-2/wal/wals/2f5bf573c8934fd3b23aa9f3761700c4
I20260620 22:59:02.598235 5393 ts_tablet_manager.cc:1918] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260620 22:59:02.598371 5393 log.cc:1196] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781996318807807-2996-0/minicluster-data/ts-1/wal/wals/2f5bf573c8934fd3b23aa9f3761700c4
I20260620 22:59:02.598423 5394 ts_tablet_manager.cc:1939] T 2f5bf573c8934fd3b23aa9f3761700c4 P 1e70e1ebf0984593bc40ce95bc7d5500: Deleting consensus metadata
I20260620 22:59:02.598843 5393 ts_tablet_manager.cc:1939] T 2f5bf573c8934fd3b23aa9f3761700c4 P a0683d16ce0b41fab23ab9d548a9509b: Deleting consensus metadata
I20260620 22:59:02.599714 4717 catalog_manager.cc:4841] TS 1e70e1ebf0984593bc40ce95bc7d5500 (127.2.237.3:40811): tablet 2f5bf573c8934fd3b23aa9f3761700c4 (table test-table [id=e2623abdf54d41d382652c73fc87f32c]) successfully deleted
I20260620 22:59:02.600011 4716 catalog_manager.cc:4841] TS a0683d16ce0b41fab23ab9d548a9509b (127.2.237.2:33969): tablet 2f5bf573c8934fd3b23aa9f3761700c4 (table test-table [id=e2623abdf54d41d382652c73fc87f32c]) successfully deleted
Jun 20 22:59:02 dist-test-slave-f63b krb5kdc[4686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996340, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.237.1@KRBTEST.COM
I20260620 22:59:02.609458 5396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:02.574424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40758 (local address 127.2.237.1:33195)
0620 22:59:02.590207 (+ 15783us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:02.590215 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:02.590244 (+ 29us) server_negotiation.cc:407] Connection header received
0620 22:59:02.590312 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:02.590319 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:02.590398 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:02.592193 (+ 1795us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:02.592671 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:02.593987 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:02.595963 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:02.596644 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:02.597486 (+ 842us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:02.603839 (+ 6353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:02.603857 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:02.603874 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:02.603915 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:02.606730 (+ 2815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:02.607475 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:02.607483 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:02.607485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:02.607622 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:02.607947 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:02.607950 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:02.607952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:02.608320 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:02.608574 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:02.608940 (+ 366us) server_negotiation.cc:299] Negotiation successful
0620 22:59:02.609143 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15597,"thread_start_us":15451,"threads_started":1}
W20260620 22:59:02.610297 4933 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40758
W20260620 22:59:02.610957 2996 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 4802: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260620 22:59:02.611188 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 4802
I20260620 22:59:02.633927 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 4942
Jun 20 22:59:02 dist-test-slave-f63b krb5kdc[4686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996340, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.237.2@KRBTEST.COM
I20260620 22:59:02.648973 5397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:02.634718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36878 (local address 127.2.237.2:33969)
0620 22:59:02.635257 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:02.635262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:02.635278 (+ 16us) server_negotiation.cc:407] Connection header received
0620 22:59:02.635331 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:02.635334 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:02.635392 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:02.635492 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:02.636712 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:02.637549 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:02.638811 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:02.639246 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:02.639349 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:02.643324 (+ 3975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:02.643338 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:02.643353 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:02.643387 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:02.646449 (+ 3062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:02.647100 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:02.647107 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:02.647109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:02.647276 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:02.647593 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:02.647596 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:02.647598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:02.647923 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:02.648195 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:02.648376 (+ 181us) server_negotiation.cc:299] Negotiation successful
0620 22:59:02.648567 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":291,"threads_started":1}
W20260620 22:59:02.649636 5072 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36878
W20260620 22:59:02.650254 2996 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 4942: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260620 22:59:02.650425 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 4942
I20260620 22:59:02.672437 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 5081
Jun 20 22:59:02 dist-test-slave-f63b krb5kdc[4686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996340, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.237.3@KRBTEST.COM
I20260620 22:59:02.686970 5398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:02.673238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56976 (local address 127.2.237.3:40811)
0620 22:59:02.673754 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:02.673758 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:02.673774 (+ 16us) server_negotiation.cc:407] Connection header received
0620 22:59:02.673832 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:02.673835 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:02.673892 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:02.673981 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:02.675271 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:02.676015 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:02.677156 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:02.677550 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:02.677638 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:02.681787 (+ 4149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:02.681802 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:02.681819 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:02.681856 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:02.684633 (+ 2777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:02.685265 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:02.685272 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:02.685274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:02.685413 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:02.685725 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:02.685728 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:02.685730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:02.686041 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:02.686235 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:02.686492 (+ 257us) server_negotiation.cc:299] Negotiation successful
0620 22:59:02.686710 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":290,"threads_started":1}
W20260620 22:59:02.687697 5211 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56976
W20260620 22:59:02.688313 2996 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 5081: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260620 22:59:02.688515 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 5081
I20260620 22:59:02.712356 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 4699
I20260620 22:59:02.713536 4771 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:02.828984 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 4699
2026-06-20T22:59:02Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8652 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 22:59:02 dist-test-slave-f63b krb5kdc[5402](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 22:59:02 dist-test-slave-f63b krb5kdc[5402](info): set up 2 sockets
Jun 20 22:59:02 dist-test-slave-f63b krb5kdc[5402](info): commencing operation
krb5kdc: starting...
W20260620 22:59:03.919214 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 22:59:04 dist-test-slave-f63b krb5kdc[5402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T22:59:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T22:59:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
I20260620 22:59:04.109563 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:39939
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:37243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:39939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:04.433192 5415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:04.433564 5415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:04.433730 5415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:04.441008 5415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 22:59:04.441164 5415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:04.441282 5415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:04.441380 5415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 22:59:04.441463 5415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 22:59:04.452158 5415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:37243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:39939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:39939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:04.453552 5415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:04.454924 5415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:04.467283 5423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:04.467314 5421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:04.467555 5420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:04.468034 5415 server_base.cc:1034] running on GCE node
I20260620 22:59:04.468878 5415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:04.470271 5415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:04.471505 5415 hybrid_clock.cc:648] HybridClock initialized: now 1781996344471466 us; error 50 us; skew 500 ppm
I20260620 22:59:04.471859 5415 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:04 dist-test-slave-f63b krb5kdc[5402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996344, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:04.474866 5415 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 22:59:04.476325 5415 webserver.cc:460] Webserver started at http://127.2.237.62:45121/ using document root <none> and password file <none>
I20260620 22:59:04.476987 5415 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:04.477126 5415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:04.477469 5415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:04.480489 5415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "9a22f19e2e954922800b1bbb33e6a5b4"
format_stamp: "Formatted at 2026-06-20 22:59:04 on dist-test-slave-f63b"
server_key: "3f96d5ab7ff8b7a1164799e9fa22ada5"
server_key_iv: "a00058c9071541d5db2c981d48ae8657"
server_key_version: "encryptionkey@0"
I20260620 22:59:04.481268 5415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "9a22f19e2e954922800b1bbb33e6a5b4"
format_stamp: "Formatted at 2026-06-20 22:59:04 on dist-test-slave-f63b"
server_key: "3f96d5ab7ff8b7a1164799e9fa22ada5"
server_key_iv: "a00058c9071541d5db2c981d48ae8657"
server_key_version: "encryptionkey@0"
I20260620 22:59:04.486294 5415 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260620 22:59:04.489665 5430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:04.491039 5415 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260620 22:59:04.491215 5415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "9a22f19e2e954922800b1bbb33e6a5b4"
format_stamp: "Formatted at 2026-06-20 22:59:04 on dist-test-slave-f63b"
server_key: "3f96d5ab7ff8b7a1164799e9fa22ada5"
server_key_iv: "a00058c9071541d5db2c981d48ae8657"
server_key_version: "encryptionkey@0"
I20260620 22:59:04.491413 5415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:04.513031 5415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:04.517160 5415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:04.517457 5415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:04.535665 5415 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:39939
I20260620 22:59:04.535694 5481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:39939 every 8 connection(s)
I20260620 22:59:04.537024 5415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 22:59:04.539454 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 5415
I20260620 22:59:04.539677 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 22:59:04.540452 2996 external_mini_cluster.cc:1428] Setting key 15bcff8155d29d8b3c6db3c3d008878f
I20260620 22:59:04.541324 5482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 20 22:59:04 dist-test-slave-f63b krb5kdc[5402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:59:04.556048 5482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a22f19e2e954922800b1bbb33e6a5b4: Bootstrap starting.
I20260620 22:59:04.560235 5482 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a22f19e2e954922800b1bbb33e6a5b4: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:04.561506 5482 log.cc:824] T 00000000000000000000000000000000 P 9a22f19e2e954922800b1bbb33e6a5b4: Log is configured to *not* fsync() on all Append() calls
I20260620 22:59:04.564318 5485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:04.542559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36336 (local address 127.2.237.62:39939)
0620 22:59:04.543376 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:04.543387 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:04.543427 (+ 40us) server_negotiation.cc:407] Connection header received
0620 22:59:04.544893 (+ 1466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:04.544912 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:04.545292 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:04.545620 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:04.547444 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:04.548768 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:04.550517 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:04.551210 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:04.551361 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:04.555995 (+ 4634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:04.556005 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:04.556030 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:04.556082 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:04.559931 (+ 3849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:04.560660 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:04.560670 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:04.560679 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:04.560878 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:04.561224 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:04.561229 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:04.561232 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:04.561927 (+ 695us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:04.562302 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:04.562665 (+ 363us) server_negotiation.cc:299] Negotiation successful
0620 22:59:04.563084 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":347,"threads_started":1}
I20260620 22:59:04.565476 5482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a22f19e2e954922800b1bbb33e6a5b4: No bootstrap required, opened a new log
I20260620 22:59:04.573087 5482 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a22f19e2e954922800b1bbb33e6a5b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a22f19e2e954922800b1bbb33e6a5b4" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 39939 } }
I20260620 22:59:04.573511 5482 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a22f19e2e954922800b1bbb33e6a5b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:04.573618 5482 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a22f19e2e954922800b1bbb33e6a5b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a22f19e2e954922800b1bbb33e6a5b4, State: Initialized, Role: FOLLOWER
I20260620 22:59:04.574239 5482 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a22f19e2e954922800b1bbb33e6a5b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a22f19e2e954922800b1bbb33e6a5b4" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 39939 } }
I20260620 22:59:04.574472 5482 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a22f19e2e954922800b1bbb33e6a5b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 22:59:04.574664 5482 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a22f19e2e954922800b1bbb33e6a5b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 22:59:04.574895 5482 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a22f19e2e954922800b1bbb33e6a5b4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:04.576979 5482 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a22f19e2e954922800b1bbb33e6a5b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a22f19e2e954922800b1bbb33e6a5b4" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 39939 } }
I20260620 22:59:04.577469 5482 leader_election.cc:302] T 00000000000000000000000000000000 P 9a22f19e2e954922800b1bbb33e6a5b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a22f19e2e954922800b1bbb33e6a5b4; no voters:
I20260620 22:59:04.578032 5482 leader_election.cc:288] T 00000000000000000000000000000000 P 9a22f19e2e954922800b1bbb33e6a5b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 22:59:04.578166 5487 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a22f19e2e954922800b1bbb33e6a5b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:59:04.578935 5487 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a22f19e2e954922800b1bbb33e6a5b4 [term 1 LEADER]: Becoming Leader. State: Replica: 9a22f19e2e954922800b1bbb33e6a5b4, State: Running, Role: LEADER
I20260620 22:59:04.579519 5487 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a22f19e2e954922800b1bbb33e6a5b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a22f19e2e954922800b1bbb33e6a5b4" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 39939 } }
I20260620 22:59:04.580372 5482 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a22f19e2e954922800b1bbb33e6a5b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 22:59:04.582510 5489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a22f19e2e954922800b1bbb33e6a5b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a22f19e2e954922800b1bbb33e6a5b4. Latest consensus state: current_term: 1 leader_uuid: "9a22f19e2e954922800b1bbb33e6a5b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a22f19e2e954922800b1bbb33e6a5b4" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 39939 } } }
I20260620 22:59:04.582738 5488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a22f19e2e954922800b1bbb33e6a5b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a22f19e2e954922800b1bbb33e6a5b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a22f19e2e954922800b1bbb33e6a5b4" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 39939 } } }
I20260620 22:59:04.582983 5489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a22f19e2e954922800b1bbb33e6a5b4 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:04.583045 5488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a22f19e2e954922800b1bbb33e6a5b4 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:04.584800 5495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 22:59:04.590026 5495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 22:59:04.599706 5495 catalog_manager.cc:1347] Generated new cluster ID: 6b921689de7d4119bfc5a134642693b6
I20260620 22:59:04.599854 5495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 22:59:04.612349 5495 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 22:59:04.614344 5495 catalog_manager.cc:1504] Loading token signing keys...
I20260620 22:59:04.624228 5495 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a22f19e2e954922800b1bbb33e6a5b4: Generated new TSK 0
I20260620 22:59:04.625355 5495 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.237.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
I20260620 22:59:04.682682 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:0
--local_ip_for_outbound_sockets=127.2.237.1
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:39939
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:37243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:05.014547 5508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:05.014845 5508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:05.015012 5508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:05.022485 5508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:05.022616 5508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:05.022830 5508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 22:59:05.032760 5508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:37243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:39939
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:05.034113 5508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:05.035512 5508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:05.048779 5513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:05.048729 5514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:05.048692 5516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:05.049628 5508 server_base.cc:1034] running on GCE node
I20260620 22:59:05.050248 5508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:05.051326 5508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:05.052541 5508 hybrid_clock.cc:648] HybridClock initialized: now 1781996345052484 us; error 46 us; skew 500 ppm
I20260620 22:59:05.052843 5508 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:05 dist-test-slave-f63b krb5kdc[5402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996345, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:05.056173 5508 init.cc:377] Logged in from keytab as kudu/127.2.237.1@KRBTEST.COM (short username kudu)
I20260620 22:59:05.057715 5508 webserver.cc:460] Webserver started at http://127.2.237.1:37355/ using document root <none> and password file <none>
I20260620 22:59:05.058427 5508 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:05.058557 5508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:05.058900 5508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:05.061738 5508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/data/instance:
uuid: "1219682eea4f48f2880ccde3b8eaf2a9"
format_stamp: "Formatted at 2026-06-20 22:59:05 on dist-test-slave-f63b"
server_key: "059dcf7f6efa100cce860dd44c9649b4"
server_key_iv: "cbd94d3cb3c82cada13f91272d568ec1"
server_key_version: "encryptionkey@0"
I20260620 22:59:05.062541 5508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance:
uuid: "1219682eea4f48f2880ccde3b8eaf2a9"
format_stamp: "Formatted at 2026-06-20 22:59:05 on dist-test-slave-f63b"
server_key: "059dcf7f6efa100cce860dd44c9649b4"
server_key_iv: "cbd94d3cb3c82cada13f91272d568ec1"
server_key_version: "encryptionkey@0"
I20260620 22:59:05.067554 5508 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260620 22:59:05.070899 5523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:05.072363 5508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260620 22:59:05.072494 5508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "1219682eea4f48f2880ccde3b8eaf2a9"
format_stamp: "Formatted at 2026-06-20 22:59:05 on dist-test-slave-f63b"
server_key: "059dcf7f6efa100cce860dd44c9649b4"
server_key_iv: "cbd94d3cb3c82cada13f91272d568ec1"
server_key_version: "encryptionkey@0"
I20260620 22:59:05.072650 5508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:05.091572 5508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:05.095536 5508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:05.095763 5508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:05.096604 5508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:59:05.097849 5508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:05.097919 5508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:05.098016 5508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:05.098059 5508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:05.132256 5508 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:33471
I20260620 22:59:05.132373 5635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:33471 every 8 connection(s)
I20260620 22:59:05.133539 5508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 22:59:05.142948 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 5508
I20260620 22:59:05.143178 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance
I20260620 22:59:05.143906 2996 external_mini_cluster.cc:1428] Setting key 2fb7e55544d03a26e4ac27fe66bc639e
Jun 20 22:59:05 dist-test-slave-f63b krb5kdc[5402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996345, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:59:05.153672 5638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:05.136037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:54775 (local address 127.2.237.62:39939)
0620 22:59:05.136497 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:05.136501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:05.137289 (+ 788us) server_negotiation.cc:407] Connection header received
0620 22:59:05.139172 (+ 1883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:05.139178 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:05.139273 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:05.139392 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:05.140899 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:05.141609 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:05.143646 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:05.144089 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:05.144196 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:05.147943 (+ 3747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:05.147948 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:05.147952 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:05.147983 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:05.150695 (+ 2712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:05.151407 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:05.151413 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:05.151416 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:05.151535 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:05.151959 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:05.151964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:05.151967 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:05.152219 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:05.152387 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:05.153082 (+ 695us) server_negotiation.cc:299] Negotiation successful
0620 22:59:05.153308 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":247,"threads_started":1}
I20260620 22:59:05.154703 5639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:05.136450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:39939 (local address 127.2.237.1:54775)
0620 22:59:05.137111 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:05.137155 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:05.138908 (+ 1753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:05.139624 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:05.139632 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:05.140133 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:05.140732 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:05.140749 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:05.141788 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:05.141792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:05.143504 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:05.143519 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:05.144330 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:05.144337 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:05.144614 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:05.145375 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:05.145401 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:05.147719 (+ 2318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:05.150837 (+ 3118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:05.150847 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:05.150866 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:05.151277 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:05.151642 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:05.151647 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:05.151651 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:05.151837 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:05.152360 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:05.152368 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:05.152714 (+ 346us) client_negotiation.cc:769] Sending connection context
0620 22:59:05.152985 (+ 271us) client_negotiation.cc:240] Negotiation successful
0620 22:59:05.153516 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":259,"threads_started":1}
I20260620 22:59:05.156352 5636 heartbeater.cc:347] Connected to a master server at 127.2.237.62:39939
I20260620 22:59:05.156713 5636 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:05.157689 5636 heartbeater.cc:511] Master 127.2.237.62:39939 requested a full tablet report, sending...
I20260620 22:59:05.160410 5447 ts_manager.cc:194] Registered new tserver with Master: 1219682eea4f48f2880ccde3b8eaf2a9 (127.2.237.1:33471)
WARNING: no policy specified for kudu/127.2.237.2@KRBTEST.COM; defaulting to no policy
I20260620 22:59:05.162765 5447 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.1@KRBTEST.COM'} at 127.2.237.1:54775
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
I20260620 22:59:05.189328 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:0
--local_ip_for_outbound_sockets=127.2.237.2
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:39939
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:37243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:05.511592 5642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:05.511902 5642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:05.512063 5642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:05.519402 5642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:05.519516 5642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:05.519716 5642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 22:59:05.529577 5642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:37243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:39939
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:05.530943 5642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:05.532240 5642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:05.545992 5650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:05.545992 5648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:05.546444 5647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:05.552296 5642 server_base.cc:1034] running on GCE node
I20260620 22:59:05.552995 5642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:05.554023 5642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:05.555249 5642 hybrid_clock.cc:648] HybridClock initialized: now 1781996345555199 us; error 52 us; skew 500 ppm
I20260620 22:59:05.555575 5642 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:05 dist-test-slave-f63b krb5kdc[5402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996345, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:05.558912 5642 init.cc:377] Logged in from keytab as kudu/127.2.237.2@KRBTEST.COM (short username kudu)
I20260620 22:59:05.560478 5642 webserver.cc:460] Webserver started at http://127.2.237.2:34165/ using document root <none> and password file <none>
I20260620 22:59:05.561201 5642 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:05.561353 5642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:05.561663 5642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:05.564622 5642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/data/instance:
uuid: "ea8b2c33ee5e481d87894976aa968472"
format_stamp: "Formatted at 2026-06-20 22:59:05 on dist-test-slave-f63b"
server_key: "893be1507da6055e2f7a8b61ca89b4a9"
server_key_iv: "691c33e11e117b22a7eaec4aa7ffe913"
server_key_version: "encryptionkey@0"
I20260620 22:59:05.565398 5642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance:
uuid: "ea8b2c33ee5e481d87894976aa968472"
format_stamp: "Formatted at 2026-06-20 22:59:05 on dist-test-slave-f63b"
server_key: "893be1507da6055e2f7a8b61ca89b4a9"
server_key_iv: "691c33e11e117b22a7eaec4aa7ffe913"
server_key_version: "encryptionkey@0"
I20260620 22:59:05.570698 5642 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260620 22:59:05.574411 5657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:05.575851 5642 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260620 22:59:05.576030 5642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "ea8b2c33ee5e481d87894976aa968472"
format_stamp: "Formatted at 2026-06-20 22:59:05 on dist-test-slave-f63b"
server_key: "893be1507da6055e2f7a8b61ca89b4a9"
server_key_iv: "691c33e11e117b22a7eaec4aa7ffe913"
server_key_version: "encryptionkey@0"
I20260620 22:59:05.576233 5642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:05.592938 5642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:05.597422 5642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:05.597729 5642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:05.598815 5642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:59:05.600114 5642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:05.600219 5642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:05.600378 5642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:05.600452 5642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:05.633816 5642 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:37195
I20260620 22:59:05.633865 5769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:37195 every 8 connection(s)
I20260620 22:59:05.635200 5642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 22:59:05.638185 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 5642
I20260620 22:59:05.638381 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance
I20260620 22:59:05.639014 2996 external_mini_cluster.cc:1428] Setting key a311cb7a578c2f740550a14be0a39e83
Jun 20 22:59:05 dist-test-slave-f63b krb5kdc[5402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996345, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.237.3@KRBTEST.COM; defaulting to no policy
I20260620 22:59:05.659078 5638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:05.637862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:34795 (local address 127.2.237.62:39939)
0620 22:59:05.638095 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:05.638101 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:05.639519 (+ 1418us) server_negotiation.cc:407] Connection header received
0620 22:59:05.641840 (+ 2321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:05.641845 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:05.641912 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:05.642002 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:05.644227 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:05.644927 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:05.647217 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:05.647653 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:05.647805 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:05.651601 (+ 3796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:05.651610 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:05.651618 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:05.651664 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:05.655362 (+ 3698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:05.656389 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:05.656398 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:05.656401 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:05.656563 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:05.657077 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:05.657083 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:05.657088 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:05.657469 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:05.657630 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:05.658387 (+ 757us) server_negotiation.cc:299] Negotiation successful
0620 22:59:05.658669 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260620 22:59:05.660055 5772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:05.638380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:39939 (local address 127.2.237.2:34795)
0620 22:59:05.639320 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:05.639370 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:05.641488 (+ 2118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:05.642303 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:05.642318 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:05.643175 (+ 857us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:05.644042 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:05.644072 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:05.645086 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:05.645094 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:05.647065 (+ 1971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:05.647077 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:05.647884 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:05.647891 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:05.648148 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:05.648898 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:05.648920 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:05.651327 (+ 2407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:05.655574 (+ 4247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:05.655584 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:05.655604 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:05.656238 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:05.656730 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:05.656734 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:05.656736 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:05.656936 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:05.657605 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:05.657612 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:05.657999 (+ 387us) client_negotiation.cc:769] Sending connection context
0620 22:59:05.658239 (+ 240us) client_negotiation.cc:240] Negotiation successful
0620 22:59:05.658839 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":358,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.3@KRBTEST.COM" created.
I20260620 22:59:05.661553 5770 heartbeater.cc:347] Connected to a master server at 127.2.237.62:39939
I20260620 22:59:05.661865 5770 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:05.662623 5770 heartbeater.cc:511] Master 127.2.237.62:39939 requested a full tablet report, sending...
I20260620 22:59:05.664026 5447 ts_manager.cc:194] Registered new tserver with Master: ea8b2c33ee5e481d87894976aa968472 (127.2.237.2:37195)
I20260620 22:59:05.664804 5447 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.2@KRBTEST.COM'} at 127.2.237.2:34795
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
I20260620 22:59:05.685355 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:0
--local_ip_for_outbound_sockets=127.2.237.3
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:39939
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:37243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:06.003921 5775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:06.004210 5775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:06.004316 5775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:06.011832 5775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:06.011951 5775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:06.012065 5775 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 22:59:06.021807 5775 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:37243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:39939
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:06.023200 5775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:06.024513 5775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:06.037534 5780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:06.037539 5783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:06.038084 5781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:06.038808 5775 server_base.cc:1034] running on GCE node
I20260620 22:59:06.039445 5775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:06.040366 5775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:06.041563 5775 hybrid_clock.cc:648] HybridClock initialized: now 1781996346041515 us; error 37 us; skew 500 ppm
I20260620 22:59:06.041883 5775 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:06 dist-test-slave-f63b krb5kdc[5402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996346, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:06.044973 5775 init.cc:377] Logged in from keytab as kudu/127.2.237.3@KRBTEST.COM (short username kudu)
I20260620 22:59:06.046414 5775 webserver.cc:460] Webserver started at http://127.2.237.3:44417/ using document root <none> and password file <none>
I20260620 22:59:06.047150 5775 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:06.047297 5775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:06.047616 5775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:06.050488 5775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/data/instance:
uuid: "ffae44c21f0d40fd9e88d47ab9af8c42"
format_stamp: "Formatted at 2026-06-20 22:59:06 on dist-test-slave-f63b"
server_key: "3692cad5934080e7129e05d44a46a5d3"
server_key_iv: "133719a789189d97a5551d84e947090f"
server_key_version: "encryptionkey@0"
I20260620 22:59:06.051329 5775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance:
uuid: "ffae44c21f0d40fd9e88d47ab9af8c42"
format_stamp: "Formatted at 2026-06-20 22:59:06 on dist-test-slave-f63b"
server_key: "3692cad5934080e7129e05d44a46a5d3"
server_key_iv: "133719a789189d97a5551d84e947090f"
server_key_version: "encryptionkey@0"
I20260620 22:59:06.056479 5775 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260620 22:59:06.059983 5790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:06.061321 5775 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 22:59:06.061506 5775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "ffae44c21f0d40fd9e88d47ab9af8c42"
format_stamp: "Formatted at 2026-06-20 22:59:06 on dist-test-slave-f63b"
server_key: "3692cad5934080e7129e05d44a46a5d3"
server_key_iv: "133719a789189d97a5551d84e947090f"
server_key_version: "encryptionkey@0"
I20260620 22:59:06.061712 5775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:06.105269 5775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:06.109423 5775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:06.109732 5775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:06.110905 5775 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:59:06.112418 5775 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:06.112525 5775 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:06.112672 5775 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:06.112756 5775 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:06.146523 5775 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:41117
I20260620 22:59:06.146638 5902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:41117 every 8 connection(s)
I20260620 22:59:06.147817 5775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 22:59:06.154196 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 5775
I20260620 22:59:06.154384 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance
I20260620 22:59:06.154865 2996 external_mini_cluster.cc:1428] Setting key 1cb8e0ffb96aaacd38b42ffe606c8ff9
Jun 20 22:59:06 dist-test-slave-f63b krb5kdc[5402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996346, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:59:06.166195 5636 heartbeater.cc:503] Master 127.2.237.62:39939 was elected leader, sending a full tablet report...
I20260620 22:59:06.168380 5638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:06.150405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:56493 (local address 127.2.237.62:39939)
0620 22:59:06.150633 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:06.150642 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:06.151870 (+ 1228us) server_negotiation.cc:407] Connection header received
0620 22:59:06.153704 (+ 1834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:06.153710 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:06.153832 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:06.153945 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:06.155696 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:06.156795 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:06.158773 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:06.159163 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:06.159294 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:06.162465 (+ 3171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:06.162470 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:06.162473 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:06.162504 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:06.165276 (+ 2772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:06.165922 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:06.165926 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:06.165928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:06.166086 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:06.166595 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:06.166600 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:06.166603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:06.166940 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:06.167099 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:06.167837 (+ 738us) server_negotiation.cc:299] Negotiation successful
0620 22:59:06.168040 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260620 22:59:06.169570 5905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:06.150967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:39939 (local address 127.2.237.3:56493)
0620 22:59:06.151655 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:06.151738 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:06.153428 (+ 1690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:06.154197 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:06.154206 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:06.154927 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:06.155539 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:06.155557 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:06.156949 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:06.156955 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:06.158562 (+ 1607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:06.158630 (+ 68us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:06.159348 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:06.159353 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:06.159551 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:06.160158 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:06.160180 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:06.162255 (+ 2075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:06.165400 (+ 3145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:06.165407 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:06.165419 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:06.165828 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:06.166221 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:06.166226 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:06.166231 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:06.166453 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:06.167072 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:06.167079 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:06.167400 (+ 321us) client_negotiation.cc:769] Sending connection context
0620 22:59:06.167738 (+ 338us) client_negotiation.cc:240] Negotiation successful
0620 22:59:06.168272 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":297,"threads_started":1}
I20260620 22:59:06.170862 5903 heartbeater.cc:347] Connected to a master server at 127.2.237.62:39939
I20260620 22:59:06.171150 5903 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:06.171828 5903 heartbeater.cc:511] Master 127.2.237.62:39939 requested a full tablet report, sending...
I20260620 22:59:06.173085 5447 ts_manager.cc:194] Registered new tserver with Master: ffae44c21f0d40fd9e88d47ab9af8c42 (127.2.237.3:41117)
I20260620 22:59:06.173949 5447 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.3@KRBTEST.COM'} at 127.2.237.3:56493
I20260620 22:59:06.182438 2996 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 22:59:06.213274 5638 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 22:59:06.210758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36340 (local address 127.2.237.62:39939)
0620 22:59:06.211001 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:06.211012 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:06.211430 (+ 418us) server_negotiation.cc:407] Connection header received
0620 22:59:06.211582 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:06.211585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:06.211646 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:06.211732 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:06.213002 (+ 1270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36340: BlockingRecv error: recv got EOF from 127.0.0.1:36340 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260620 22:59:06.216602 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 5508
W20260620 22:59:06.220224 2996 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 5508: Not authorized: Client connection negotiation failed: client connection to 127.2.237.1:33471: server requires authentication, but client does not have Kerberos credentials available
I20260620 22:59:06.220443 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 5508
I20260620 22:59:06.239185 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 5642
W20260620 22:59:06.241715 2996 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 5642: Not authorized: Client connection negotiation failed: client connection to 127.2.237.2:37195: server requires authentication, but client does not have Kerberos credentials available
I20260620 22:59:06.241927 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 5642
I20260620 22:59:06.257378 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 5775
W20260620 22:59:06.260385 2996 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 5775: Not authorized: Client connection negotiation failed: client connection to 127.2.237.3:41117: server requires authentication, but client does not have Kerberos credentials available
I20260620 22:59:06.260504 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 5775
I20260620 22:59:06.275971 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 5415
I20260620 22:59:06.277179 5477 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:06.378475 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 5415
2026-06-20T22:59:06Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3546 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 22:59:06 dist-test-slave-f63b krb5kdc[5920](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 22:59:06 dist-test-slave-f63b krb5kdc[5920](info): set up 2 sockets
Jun 20 22:59:06 dist-test-slave-f63b krb5kdc[5920](info): commencing operation
krb5kdc: starting...
W20260620 22:59:07.471256 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.000s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 22:59:07 dist-test-slave-f63b krb5kdc[5920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T22:59:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T22:59:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
I20260620 22:59:07.665438 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:36899
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:46177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:36899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:07.987385 5933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:07.987706 5933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:07.987865 5933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:07.995168 5933 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 22:59:07.995280 5933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:07.995393 5933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:07.995500 5933 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 22:59:07.995585 5933 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 22:59:08.005604 5933 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:46177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:36899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:36899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:08.007090 5933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:08.008450 5933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:08.021003 5941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:08.021193 5939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:08.021215 5938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:08.022001 5933 server_base.cc:1034] running on GCE node
I20260620 22:59:08.022858 5933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:08.024291 5933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:08.025521 5933 hybrid_clock.cc:648] HybridClock initialized: now 1781996348025465 us; error 51 us; skew 500 ppm
I20260620 22:59:08.025879 5933 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:08 dist-test-slave-f63b krb5kdc[5920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996348, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:08.032430 5933 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 22:59:08.033958 5933 webserver.cc:460] Webserver started at http://127.2.237.62:46785/ using document root <none> and password file <none>
I20260620 22:59:08.034708 5933 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:08.034814 5933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:08.035073 5933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:08.038074 5933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "186ec109233e4dffba562fbe3f7d59d3"
format_stamp: "Formatted at 2026-06-20 22:59:08 on dist-test-slave-f63b"
server_key: "e524f6388e270d44b8302b78b4237296"
server_key_iv: "394c1daeaa3648887bc181a3f82d8eb9"
server_key_version: "encryptionkey@0"
I20260620 22:59:08.038882 5933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "186ec109233e4dffba562fbe3f7d59d3"
format_stamp: "Formatted at 2026-06-20 22:59:08 on dist-test-slave-f63b"
server_key: "e524f6388e270d44b8302b78b4237296"
server_key_iv: "394c1daeaa3648887bc181a3f82d8eb9"
server_key_version: "encryptionkey@0"
I20260620 22:59:08.044040 5933 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260620 22:59:08.047380 5948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:08.048750 5933 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 22:59:08.048935 5933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "186ec109233e4dffba562fbe3f7d59d3"
format_stamp: "Formatted at 2026-06-20 22:59:08 on dist-test-slave-f63b"
server_key: "e524f6388e270d44b8302b78b4237296"
server_key_iv: "394c1daeaa3648887bc181a3f82d8eb9"
server_key_version: "encryptionkey@0"
I20260620 22:59:08.049111 5933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:08.068677 5933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:08.072883 5933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:08.073186 5933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:08.091745 5933 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:36899
I20260620 22:59:08.091843 5999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:36899 every 8 connection(s)
I20260620 22:59:08.093153 5933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 22:59:08.093918 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 5933
I20260620 22:59:08.094225 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 22:59:08.095016 2996 external_mini_cluster.cc:1428] Setting key cf0edc12a40d276e921a01529e0958bc
I20260620 22:59:08.097649 6000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 20 22:59:08 dist-test-slave-f63b krb5kdc[5920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:59:08.110972 6000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 186ec109233e4dffba562fbe3f7d59d3: Bootstrap starting.
I20260620 22:59:08.115144 6000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 186ec109233e4dffba562fbe3f7d59d3: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:08.116443 6000 log.cc:824] T 00000000000000000000000000000000 P 186ec109233e4dffba562fbe3f7d59d3: Log is configured to *not* fsync() on all Append() calls
I20260620 22:59:08.118826 6003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:08.096833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51852 (local address 127.2.237.62:36899)
0620 22:59:08.097613 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:08.097625 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:08.097670 (+ 45us) server_negotiation.cc:407] Connection header received
0620 22:59:08.099374 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:08.099395 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:08.099757 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:08.100103 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:08.101881 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:08.103320 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:08.105059 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:08.105690 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:08.105826 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:08.110623 (+ 4797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:08.110635 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:08.110656 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:08.110703 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:08.114474 (+ 3771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:08.115205 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:08.115214 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:08.115223 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:08.115442 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:08.115820 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:08.115825 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:08.115828 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:08.116560 (+ 732us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:08.116824 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:08.117361 (+ 537us) server_negotiation.cc:299] Negotiation successful
0620 22:59:08.117755 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":371,"threads_started":1}
I20260620 22:59:08.119571 6000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 186ec109233e4dffba562fbe3f7d59d3: No bootstrap required, opened a new log
I20260620 22:59:08.122428 2996 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 22:59:08.127015 6000 raft_consensus.cc:348] T 00000000000000000000000000000000 P 186ec109233e4dffba562fbe3f7d59d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186ec109233e4dffba562fbe3f7d59d3" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36899 } }
I20260620 22:59:08.127403 6000 raft_consensus.cc:374] T 00000000000000000000000000000000 P 186ec109233e4dffba562fbe3f7d59d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:08.127509 6000 raft_consensus.cc:729] T 00000000000000000000000000000000 P 186ec109233e4dffba562fbe3f7d59d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 186ec109233e4dffba562fbe3f7d59d3, State: Initialized, Role: FOLLOWER
I20260620 22:59:08.128149 6000 consensus_queue.cc:260] T 00000000000000000000000000000000 P 186ec109233e4dffba562fbe3f7d59d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186ec109233e4dffba562fbe3f7d59d3" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36899 } }
I20260620 22:59:08.128429 6000 raft_consensus.cc:388] T 00000000000000000000000000000000 P 186ec109233e4dffba562fbe3f7d59d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 22:59:08.128573 6000 raft_consensus.cc:482] T 00000000000000000000000000000000 P 186ec109233e4dffba562fbe3f7d59d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 22:59:08.128757 6000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 186ec109233e4dffba562fbe3f7d59d3 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:08.130960 6000 raft_consensus.cc:504] T 00000000000000000000000000000000 P 186ec109233e4dffba562fbe3f7d59d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186ec109233e4dffba562fbe3f7d59d3" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36899 } }
I20260620 22:59:08.131466 6000 leader_election.cc:302] T 00000000000000000000000000000000 P 186ec109233e4dffba562fbe3f7d59d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 186ec109233e4dffba562fbe3f7d59d3; no voters:
I20260620 22:59:08.132131 6000 leader_election.cc:288] T 00000000000000000000000000000000 P 186ec109233e4dffba562fbe3f7d59d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 22:59:08.132236 6006 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 186ec109233e4dffba562fbe3f7d59d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:59:08.133409 6006 raft_consensus.cc:686] T 00000000000000000000000000000000 P 186ec109233e4dffba562fbe3f7d59d3 [term 1 LEADER]: Becoming Leader. State: Replica: 186ec109233e4dffba562fbe3f7d59d3, State: Running, Role: LEADER
I20260620 22:59:08.134156 6006 consensus_queue.cc:237] T 00000000000000000000000000000000 P 186ec109233e4dffba562fbe3f7d59d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186ec109233e4dffba562fbe3f7d59d3" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36899 } }
I20260620 22:59:08.134852 6000 sys_catalog.cc:564] T 00000000000000000000000000000000 P 186ec109233e4dffba562fbe3f7d59d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 22:59:08.137132 6008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 186ec109233e4dffba562fbe3f7d59d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 186ec109233e4dffba562fbe3f7d59d3. Latest consensus state: current_term: 1 leader_uuid: "186ec109233e4dffba562fbe3f7d59d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186ec109233e4dffba562fbe3f7d59d3" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36899 } } }
I20260620 22:59:08.137503 6008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 186ec109233e4dffba562fbe3f7d59d3 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:08.138065 6007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 186ec109233e4dffba562fbe3f7d59d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "186ec109233e4dffba562fbe3f7d59d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "186ec109233e4dffba562fbe3f7d59d3" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36899 } } }
I20260620 22:59:08.138363 6007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 186ec109233e4dffba562fbe3f7d59d3 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:08.140964 6012 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 22:59:08.147569 6012 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 22:59:08.162220 6012 catalog_manager.cc:1347] Generated new cluster ID: bbdcea6a747f450f92c2bc921b8ddd3a
I20260620 22:59:08.162341 6012 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 22:59:08.177557 6003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:08.167632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51864 (local address 127.2.237.62:36899)
0620 22:59:08.167798 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:08.167805 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:08.168525 (+ 720us) server_negotiation.cc:407] Connection header received
0620 22:59:08.168704 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:08.168710 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:08.168796 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:08.168916 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:08.170077 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:08.171277 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:08.173019 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:08.173701 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:08.173834 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:08.175152 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:08.175159 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:08.175164 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 22:59:08.176060 (+ 896us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:08.176115 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 22:59:08.176151 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:08.176275 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:08.176441 (+ 166us) server_negotiation.cc:299] Negotiation successful
0620 22:59:08.176539 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 22:59:08.178020 6012 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 22:59:08.179895 6012 catalog_manager.cc:1504] Loading token signing keys...
I20260620 22:59:08.189934 6012 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 186ec109233e4dffba562fbe3f7d59d3: Generated new TSK 0
I20260620 22:59:08.190935 6012 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 22:59:08.205173 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 5933
I20260620 22:59:08.206398 5995 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:08.300364 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 5933
2026-06-20T22:59:08Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1916 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 22:59:08 dist-test-slave-f63b krb5kdc[6034](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 22:59:08 dist-test-slave-f63b krb5kdc[6034](info): set up 2 sockets
Jun 20 22:59:08 dist-test-slave-f63b krb5kdc[6034](info): commencing operation
krb5kdc: starting...
W20260620 22:59:09.391059 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.008s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 22:59:09 dist-test-slave-f63b krb5kdc[6034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T22:59:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T22:59:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
I20260620 22:59:09.587833 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:33925
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:35877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:33925
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:09.927834 6047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:09.928109 6047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:09.928283 6047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:09.935314 6047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 22:59:09.935413 6047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:09.935496 6047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:09.935586 6047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 22:59:09.935670 6047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 22:59:09.945401 6047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:35877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:33925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:33925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:09.946817 6047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:09.948134 6047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:09.959867 6052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:09.959905 6055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:09.959933 6053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:09.960636 6047 server_base.cc:1034] running on GCE node
I20260620 22:59:09.961436 6047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:09.962872 6047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:09.964092 6047 hybrid_clock.cc:648] HybridClock initialized: now 1781996349964048 us; error 38 us; skew 500 ppm
I20260620 22:59:09.964449 6047 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:09 dist-test-slave-f63b krb5kdc[6034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996349, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:09.967590 6047 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 22:59:09.969111 6047 webserver.cc:460] Webserver started at http://127.2.237.62:40447/ using document root <none> and password file <none>
I20260620 22:59:09.969789 6047 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:09.969931 6047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:09.970229 6047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:09.973198 6047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "2ccdfe6826244fb48f3a40b2f0e0e5f8"
format_stamp: "Formatted at 2026-06-20 22:59:09 on dist-test-slave-f63b"
server_key: "8f5b0eefd21657f5231c73309857eaf5"
server_key_iv: "c49e8ba2e2e2ecc318681c66048404fd"
server_key_version: "encryptionkey@0"
I20260620 22:59:09.974007 6047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "2ccdfe6826244fb48f3a40b2f0e0e5f8"
format_stamp: "Formatted at 2026-06-20 22:59:09 on dist-test-slave-f63b"
server_key: "8f5b0eefd21657f5231c73309857eaf5"
server_key_iv: "c49e8ba2e2e2ecc318681c66048404fd"
server_key_version: "encryptionkey@0"
I20260620 22:59:09.979346 6047 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260620 22:59:09.982892 6062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:09.984288 6047 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260620 22:59:09.984489 6047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "2ccdfe6826244fb48f3a40b2f0e0e5f8"
format_stamp: "Formatted at 2026-06-20 22:59:09 on dist-test-slave-f63b"
server_key: "8f5b0eefd21657f5231c73309857eaf5"
server_key_iv: "c49e8ba2e2e2ecc318681c66048404fd"
server_key_version: "encryptionkey@0"
I20260620 22:59:09.984694 6047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:10.007413 6047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:10.011389 6047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:10.011682 6047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:10.029613 6047 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:33925
I20260620 22:59:10.029636 6113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:33925 every 8 connection(s)
I20260620 22:59:10.031003 6047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 22:59:10.034636 6114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:10.036188 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 6047
I20260620 22:59:10.036418 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 22:59:10.037222 2996 external_mini_cluster.cc:1428] Setting key a57124c5f83c7ddf0936591ab27dc0df
I20260620 22:59:10.048805 6114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ccdfe6826244fb48f3a40b2f0e0e5f8: Bootstrap starting.
Jun 20 22:59:10 dist-test-slave-f63b krb5kdc[6034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:59:10.052134 6114 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ccdfe6826244fb48f3a40b2f0e0e5f8: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:10.053164 6114 log.cc:824] T 00000000000000000000000000000000 P 2ccdfe6826244fb48f3a40b2f0e0e5f8: Log is configured to *not* fsync() on all Append() calls
I20260620 22:59:10.056303 6114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ccdfe6826244fb48f3a40b2f0e0e5f8: No bootstrap required, opened a new log
I20260620 22:59:10.060197 6117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:10.039284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57982 (local address 127.2.237.62:33925)
0620 22:59:10.040255 (+ 971us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:10.040267 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:10.040313 (+ 46us) server_negotiation.cc:407] Connection header received
0620 22:59:10.041799 (+ 1486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:10.041825 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:10.042212 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:10.042523 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:10.044217 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:10.045539 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:10.047391 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:10.048128 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:10.048267 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:10.052756 (+ 4489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:10.052764 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:10.052779 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:10.052820 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:10.055987 (+ 3167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:10.056815 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:10.056822 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:10.056828 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:10.057009 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:10.057399 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:10.057408 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:10.057410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:10.057936 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:10.058295 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:10.058737 (+ 442us) server_negotiation.cc:299] Negotiation successful
0620 22:59:10.059170 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":760,"thread_start_us":459,"threads_started":1}
I20260620 22:59:10.064401 6114 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ccdfe6826244fb48f3a40b2f0e0e5f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ccdfe6826244fb48f3a40b2f0e0e5f8" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 33925 } }
I20260620 22:59:10.064770 6114 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ccdfe6826244fb48f3a40b2f0e0e5f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:10.064883 6114 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ccdfe6826244fb48f3a40b2f0e0e5f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ccdfe6826244fb48f3a40b2f0e0e5f8, State: Initialized, Role: FOLLOWER
I20260620 22:59:10.065485 6114 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ccdfe6826244fb48f3a40b2f0e0e5f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ccdfe6826244fb48f3a40b2f0e0e5f8" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 33925 } }
I20260620 22:59:10.065793 6114 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ccdfe6826244fb48f3a40b2f0e0e5f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 22:59:10.065938 6114 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ccdfe6826244fb48f3a40b2f0e0e5f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 22:59:10.066133 6114 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ccdfe6826244fb48f3a40b2f0e0e5f8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:10.068495 6114 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ccdfe6826244fb48f3a40b2f0e0e5f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ccdfe6826244fb48f3a40b2f0e0e5f8" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 33925 } }
I20260620 22:59:10.069061 6114 leader_election.cc:302] T 00000000000000000000000000000000 P 2ccdfe6826244fb48f3a40b2f0e0e5f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ccdfe6826244fb48f3a40b2f0e0e5f8; no voters:
I20260620 22:59:10.069684 6114 leader_election.cc:288] T 00000000000000000000000000000000 P 2ccdfe6826244fb48f3a40b2f0e0e5f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 22:59:10.069815 6119 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ccdfe6826244fb48f3a40b2f0e0e5f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:59:10.070612 6119 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ccdfe6826244fb48f3a40b2f0e0e5f8 [term 1 LEADER]: Becoming Leader. State: Replica: 2ccdfe6826244fb48f3a40b2f0e0e5f8, State: Running, Role: LEADER
I20260620 22:59:10.071187 6119 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ccdfe6826244fb48f3a40b2f0e0e5f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ccdfe6826244fb48f3a40b2f0e0e5f8" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 33925 } }
I20260620 22:59:10.072019 6114 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ccdfe6826244fb48f3a40b2f0e0e5f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 22:59:10.074008 6120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ccdfe6826244fb48f3a40b2f0e0e5f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ccdfe6826244fb48f3a40b2f0e0e5f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ccdfe6826244fb48f3a40b2f0e0e5f8" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 33925 } } }
I20260620 22:59:10.074383 6120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ccdfe6826244fb48f3a40b2f0e0e5f8 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:10.074019 6121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ccdfe6826244fb48f3a40b2f0e0e5f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ccdfe6826244fb48f3a40b2f0e0e5f8. Latest consensus state: current_term: 1 leader_uuid: "2ccdfe6826244fb48f3a40b2f0e0e5f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ccdfe6826244fb48f3a40b2f0e0e5f8" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 33925 } } }
I20260620 22:59:10.075460 6121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ccdfe6826244fb48f3a40b2f0e0e5f8 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:10.076937 6124 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 22:59:10.081171 6124 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 22:59:10.089787 6124 catalog_manager.cc:1347] Generated new cluster ID: 4c0f054324f441e9be2e8846b4a290c9
I20260620 22:59:10.089897 6124 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 22:59:10.100553 6124 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 22:59:10.101790 6124 catalog_manager.cc:1504] Loading token signing keys...
I20260620 22:59:10.121387 6124 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ccdfe6826244fb48f3a40b2f0e0e5f8: Generated new TSK 0
I20260620 22:59:10.122351 6124 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.237.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
I20260620 22:59:10.173679 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:0
--local_ip_for_outbound_sockets=127.2.237.1
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:33925
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:35877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:10.499064 6140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:10.499346 6140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:10.499537 6140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:10.506798 6140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:10.506920 6140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:10.507076 6140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 22:59:10.517117 6140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:35877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:33925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:10.518429 6140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:10.519759 6140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:10.532840 6145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:10.532953 6146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:10.533543 6148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:10.533716 6140 server_base.cc:1034] running on GCE node
I20260620 22:59:10.534348 6140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:10.535277 6140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:10.536470 6140 hybrid_clock.cc:648] HybridClock initialized: now 1781996350536421 us; error 57 us; skew 500 ppm
I20260620 22:59:10.536789 6140 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:10 dist-test-slave-f63b krb5kdc[6034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996350, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:10.539862 6140 init.cc:377] Logged in from keytab as kudu/127.2.237.1@KRBTEST.COM (short username kudu)
I20260620 22:59:10.541365 6140 webserver.cc:460] Webserver started at http://127.2.237.1:46773/ using document root <none> and password file <none>
I20260620 22:59:10.542053 6140 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:10.542188 6140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:10.542496 6140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:10.545382 6140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/data/instance:
uuid: "f0ab7ddddba64527a4f3f3abf3969e88"
format_stamp: "Formatted at 2026-06-20 22:59:10 on dist-test-slave-f63b"
server_key: "2b5ba90632d1dc8e9d6cdaecf578d33d"
server_key_iv: "52737c45c3155f36137cfc70c4ce6211"
server_key_version: "encryptionkey@0"
I20260620 22:59:10.546149 6140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance:
uuid: "f0ab7ddddba64527a4f3f3abf3969e88"
format_stamp: "Formatted at 2026-06-20 22:59:10 on dist-test-slave-f63b"
server_key: "2b5ba90632d1dc8e9d6cdaecf578d33d"
server_key_iv: "52737c45c3155f36137cfc70c4ce6211"
server_key_version: "encryptionkey@0"
I20260620 22:59:10.551388 6140 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260620 22:59:10.555219 6155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:10.556635 6140 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260620 22:59:10.556851 6140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "f0ab7ddddba64527a4f3f3abf3969e88"
format_stamp: "Formatted at 2026-06-20 22:59:10 on dist-test-slave-f63b"
server_key: "2b5ba90632d1dc8e9d6cdaecf578d33d"
server_key_iv: "52737c45c3155f36137cfc70c4ce6211"
server_key_version: "encryptionkey@0"
I20260620 22:59:10.557029 6140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:10.585896 6140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:10.590637 6140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:10.590965 6140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:10.592021 6140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:59:10.593531 6140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:10.593613 6140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:10.593788 6140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:10.593856 6140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:10.628486 6140 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:35885
I20260620 22:59:10.628528 6267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:35885 every 8 connection(s)
I20260620 22:59:10.629813 6140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 22:59:10.632530 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 6140
I20260620 22:59:10.632727 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance
I20260620 22:59:10.633725 2996 external_mini_cluster.cc:1428] Setting key 0171832c18fbf6a4b746f0c6df52f917
Jun 20 22:59:10 dist-test-slave-f63b krb5kdc[6034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996350, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:59:10.653600 6270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:10.632638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:49009 (local address 127.2.237.62:33925)
0620 22:59:10.634827 (+ 2189us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:10.634836 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:10.634860 (+ 24us) server_negotiation.cc:407] Connection header received
0620 22:59:10.636860 (+ 2000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:10.636869 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:10.636964 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:10.637077 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:10.638841 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:10.640032 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:10.641945 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:10.642395 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:10.642568 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:10.646467 (+ 3899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:10.646476 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:10.646484 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:10.646529 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:10.650077 (+ 3548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:10.650940 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:10.650949 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:10.650955 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:10.651115 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:10.651572 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:10.651577 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:10.651580 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:10.651959 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:10.652123 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:10.652898 (+ 775us) server_negotiation.cc:299] Negotiation successful
0620 22:59:10.653148 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2063,"thread_start_us":1967,"threads_started":1}
WARNING: no policy specified for kudu/127.2.237.2@KRBTEST.COM; defaulting to no policy
I20260620 22:59:10.654419 6271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:10.633258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:33925 (local address 127.2.237.1:49009)
0620 22:59:10.634235 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:10.634284 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:10.636568 (+ 2284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:10.637359 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:10.637369 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:10.637966 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:10.638662 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:10.638685 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:10.640214 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:10.640219 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:10.641750 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:10.641764 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:10.642696 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:10.642706 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:10.642978 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:10.643713 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:10.643764 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:10.646222 (+ 2458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:10.650220 (+ 3998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:10.650227 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:10.650254 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:10.650800 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:10.651251 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:10.651255 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:10.651257 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:10.651445 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:10.652090 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:10.652096 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:10.652477 (+ 381us) client_negotiation.cc:769] Sending connection context
0620 22:59:10.652733 (+ 256us) client_negotiation.cc:240] Negotiation successful
0620 22:59:10.653161 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"thread_start_us":397,"threads_started":1}
I20260620 22:59:10.656165 6268 heartbeater.cc:347] Connected to a master server at 127.2.237.62:33925
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.2@KRBTEST.COM" created.
I20260620 22:59:10.656562 6268 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:10.657497 6268 heartbeater.cc:511] Master 127.2.237.62:33925 requested a full tablet report, sending...
I20260620 22:59:10.660005 6079 ts_manager.cc:194] Registered new tserver with Master: f0ab7ddddba64527a4f3f3abf3969e88 (127.2.237.1:35885)
I20260620 22:59:10.662186 6079 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.1@KRBTEST.COM'} at 127.2.237.1:49009
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
I20260620 22:59:10.684000 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:0
--local_ip_for_outbound_sockets=127.2.237.2
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:33925
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:35877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:11.008514 6274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:11.008821 6274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:11.008919 6274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:11.016240 6274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:11.016356 6274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:11.016472 6274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 22:59:11.026296 6274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:35877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:33925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:11.027588 6274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:11.028836 6274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:11.043475 6280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:11.043538 6279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:11.044035 6282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:11.044945 6274 server_base.cc:1034] running on GCE node
I20260620 22:59:11.045646 6274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:11.046782 6274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:11.048015 6274 hybrid_clock.cc:648] HybridClock initialized: now 1781996351047943 us; error 58 us; skew 500 ppm
I20260620 22:59:11.048388 6274 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:11 dist-test-slave-f63b krb5kdc[6034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996351, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:11.051738 6274 init.cc:377] Logged in from keytab as kudu/127.2.237.2@KRBTEST.COM (short username kudu)
I20260620 22:59:11.053272 6274 webserver.cc:460] Webserver started at http://127.2.237.2:46055/ using document root <none> and password file <none>
I20260620 22:59:11.054003 6274 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:11.054136 6274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:11.054466 6274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:11.057451 6274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/data/instance:
uuid: "63acdd4852004f25bad884445f413b2c"
format_stamp: "Formatted at 2026-06-20 22:59:11 on dist-test-slave-f63b"
server_key: "5ec902145299e84e1ab66cb383bcac64"
server_key_iv: "1ffe0f659b694cc0cb19981beb115778"
server_key_version: "encryptionkey@0"
I20260620 22:59:11.058260 6274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance:
uuid: "63acdd4852004f25bad884445f413b2c"
format_stamp: "Formatted at 2026-06-20 22:59:11 on dist-test-slave-f63b"
server_key: "5ec902145299e84e1ab66cb383bcac64"
server_key_iv: "1ffe0f659b694cc0cb19981beb115778"
server_key_version: "encryptionkey@0"
I20260620 22:59:11.063750 6274 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260620 22:59:11.067221 6289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:11.068601 6274 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 22:59:11.068826 6274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "63acdd4852004f25bad884445f413b2c"
format_stamp: "Formatted at 2026-06-20 22:59:11 on dist-test-slave-f63b"
server_key: "5ec902145299e84e1ab66cb383bcac64"
server_key_iv: "1ffe0f659b694cc0cb19981beb115778"
server_key_version: "encryptionkey@0"
I20260620 22:59:11.069038 6274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:11.093865 6274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:11.097868 6274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:11.098192 6274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:11.099225 6274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:59:11.100497 6274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:11.100566 6274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:11.100740 6274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:11.100790 6274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:11.133634 6274 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:44407
I20260620 22:59:11.133682 6401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:44407 every 8 connection(s)
I20260620 22:59:11.134974 6274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 22:59:11.142633 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 6274
I20260620 22:59:11.142872 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance
I20260620 22:59:11.143388 2996 external_mini_cluster.cc:1428] Setting key 74e3283e78b3c264309c4699a996864e
Jun 20 22:59:11 dist-test-slave-f63b krb5kdc[6034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996351, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:59:11.157217 6270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:11.138002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:42639 (local address 127.2.237.62:33925)
0620 22:59:11.138230 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:11.138238 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:11.139486 (+ 1248us) server_negotiation.cc:407] Connection header received
0620 22:59:11.141328 (+ 1842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:11.141333 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:11.141407 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:11.141506 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:11.143400 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:11.144507 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:11.146429 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:11.147242 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:11.147411 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:11.151283 (+ 3872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:11.151288 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:11.151291 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:11.151323 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:11.154120 (+ 2797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:11.154919 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:11.154927 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:11.154932 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:11.155063 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:11.155509 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:11.155514 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:11.155517 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:11.155780 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:11.155945 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:11.156651 (+ 706us) server_negotiation.cc:299] Negotiation successful
0620 22:59:11.156875 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260620 22:59:11.158260 6404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:11.138500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:33925 (local address 127.2.237.2:42639)
0620 22:59:11.139313 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:11.139351 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:11.141044 (+ 1693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:11.141774 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:11.141789 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:11.142353 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:11.143200 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:11.143221 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:11.144660 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:11.144665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:11.146267 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:11.146278 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:11.147481 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:11.147486 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:11.147692 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:11.148478 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:11.148500 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:11.151045 (+ 2545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:11.154277 (+ 3232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:11.154286 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:11.154304 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:11.154795 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:11.155193 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:11.155205 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:11.155208 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:11.155390 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:11.155917 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:11.155926 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:11.156286 (+ 360us) client_negotiation.cc:769] Sending connection context
0620 22:59:11.156550 (+ 264us) client_negotiation.cc:240] Negotiation successful
0620 22:59:11.157089 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":301,"threads_started":1}
I20260620 22:59:11.159802 6402 heartbeater.cc:347] Connected to a master server at 127.2.237.62:33925
I20260620 22:59:11.160195 6402 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:11.161259 6402 heartbeater.cc:511] Master 127.2.237.62:33925 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.2.237.3@KRBTEST.COM; defaulting to no policy
I20260620 22:59:11.163089 6079 ts_manager.cc:194] Registered new tserver with Master: 63acdd4852004f25bad884445f413b2c (127.2.237.2:44407)
I20260620 22:59:11.163895 6079 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.2@KRBTEST.COM'} at 127.2.237.2:42639
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
I20260620 22:59:11.190459 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:0
--local_ip_for_outbound_sockets=127.2.237.3
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:33925
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:35877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:11.512578 6407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:11.512861 6407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:11.512952 6407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:11.520519 6407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:11.520633 6407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:11.520748 6407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 22:59:11.531008 6407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:35877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:33925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:11.532258 6407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:11.533583 6407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:11.546382 6412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:11.546489 6415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:11.546521 6413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:11.547603 6407 server_base.cc:1034] running on GCE node
I20260620 22:59:11.548166 6407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:11.549058 6407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:11.550280 6407 hybrid_clock.cc:648] HybridClock initialized: now 1781996351550251 us; error 48 us; skew 500 ppm
I20260620 22:59:11.550654 6407 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:11 dist-test-slave-f63b krb5kdc[6034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996351, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:11.553769 6407 init.cc:377] Logged in from keytab as kudu/127.2.237.3@KRBTEST.COM (short username kudu)
I20260620 22:59:11.555301 6407 webserver.cc:460] Webserver started at http://127.2.237.3:41291/ using document root <none> and password file <none>
I20260620 22:59:11.555989 6407 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:11.556128 6407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:11.556432 6407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:11.559450 6407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/data/instance:
uuid: "1e06273ed58847b6b10dea2d4601e597"
format_stamp: "Formatted at 2026-06-20 22:59:11 on dist-test-slave-f63b"
server_key: "77a1bf85e69cf3d4ca92ac1801c25329"
server_key_iv: "b38a75c17dca86464a356c646f15e00e"
server_key_version: "encryptionkey@0"
I20260620 22:59:11.560217 6407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance:
uuid: "1e06273ed58847b6b10dea2d4601e597"
format_stamp: "Formatted at 2026-06-20 22:59:11 on dist-test-slave-f63b"
server_key: "77a1bf85e69cf3d4ca92ac1801c25329"
server_key_iv: "b38a75c17dca86464a356c646f15e00e"
server_key_version: "encryptionkey@0"
I20260620 22:59:11.565241 6407 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260620 22:59:11.568835 6422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:11.570097 6407 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 22:59:11.570266 6407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "1e06273ed58847b6b10dea2d4601e597"
format_stamp: "Formatted at 2026-06-20 22:59:11 on dist-test-slave-f63b"
server_key: "77a1bf85e69cf3d4ca92ac1801c25329"
server_key_iv: "b38a75c17dca86464a356c646f15e00e"
server_key_version: "encryptionkey@0"
I20260620 22:59:11.570467 6407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:11.601516 6407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:11.605965 6407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:11.606272 6407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:11.607349 6407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:59:11.608693 6407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:11.608801 6407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:11.608949 6407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:11.609030 6407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:11.642541 6407 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:35763
I20260620 22:59:11.642601 6534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:35763 every 8 connection(s)
I20260620 22:59:11.643832 6407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 22:59:11.649458 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 6407
I20260620 22:59:11.649667 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance
I20260620 22:59:11.650147 2996 external_mini_cluster.cc:1428] Setting key 5d8b95afccb6d9fee0b886322be87903
Jun 20 22:59:11 dist-test-slave-f63b krb5kdc[6034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996351, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:59:11.663681 6270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:11.646313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:54877 (local address 127.2.237.62:33925)
0620 22:59:11.646504 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:11.646509 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:11.647793 (+ 1284us) server_negotiation.cc:407] Connection header received
0620 22:59:11.649663 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:11.649680 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:11.649805 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:11.649917 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:11.651752 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:11.652819 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:11.654382 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:11.654843 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:11.654973 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:11.658174 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:11.658187 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:11.658190 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:11.658221 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:11.660648 (+ 2427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:11.661424 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:11.661430 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:11.661433 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:11.661558 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:11.661969 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:11.661976 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:11.661979 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:11.662242 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:11.662411 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:11.663111 (+ 700us) server_negotiation.cc:299] Negotiation successful
0620 22:59:11.663315 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260620 22:59:11.664824 6537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:11.646911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:33925 (local address 127.2.237.3:54877)
0620 22:59:11.647604 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:11.647658 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:11.649345 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:11.650150 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:11.650158 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:11.650809 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:11.651592 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:11.651613 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:11.652962 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:11.652969 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:11.654246 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:11.654257 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:11.655070 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:11.655077 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:11.655306 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:11.655873 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:11.655894 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:11.657976 (+ 2082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:11.660805 (+ 2829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:11.660813 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:11.660832 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:11.661306 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:11.661667 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:11.661672 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:11.661677 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:11.661857 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:11.662384 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:11.662392 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:11.662767 (+ 375us) client_negotiation.cc:769] Sending connection context
0620 22:59:11.663026 (+ 259us) client_negotiation.cc:240] Negotiation successful
0620 22:59:11.663538 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":298,"threads_started":1}
I20260620 22:59:11.665131 6268 heartbeater.cc:503] Master 127.2.237.62:33925 was elected leader, sending a full tablet report...
I20260620 22:59:11.666458 6535 heartbeater.cc:347] Connected to a master server at 127.2.237.62:33925
I20260620 22:59:11.666802 6535 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:11.667457 6535 heartbeater.cc:511] Master 127.2.237.62:33925 requested a full tablet report, sending...
I20260620 22:59:11.668743 6078 ts_manager.cc:194] Registered new tserver with Master: 1e06273ed58847b6b10dea2d4601e597 (127.2.237.3:35763)
I20260620 22:59:11.669559 6078 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.3@KRBTEST.COM'} at 127.2.237.3:54877
I20260620 22:59:11.677327 2996 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 22:59:11 dist-test-slave-f63b krb5kdc[6034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996351, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 20 22:59:11 dist-test-slave-f63b krb5kdc[6034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996351, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:59:11.724220 6270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:11.710438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57998 (local address 127.2.237.62:33925)
0620 22:59:11.710778 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:11.710782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:11.711152 (+ 370us) server_negotiation.cc:407] Connection header received
0620 22:59:11.711312 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:11.711316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:11.711388 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:11.711483 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:11.713009 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:11.713723 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:11.715003 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:11.715391 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:11.715483 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:11.719422 (+ 3939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:11.719427 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:11.719430 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:11.719460 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:11.721976 (+ 2516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:11.722646 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:11.722650 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:11.722653 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:11.722783 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:11.723110 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:11.723113 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:11.723116 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:11.723420 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:11.723597 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:11.723854 (+ 257us) server_negotiation.cc:299] Negotiation successful
0620 22:59:11.724004 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260620 22:59:11.724812 6078 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:57998
I20260620 22:59:11.726305 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 6140
Jun 20 22:59:11 dist-test-slave-f63b krb5kdc[6034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996351, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.2.237.1@KRBTEST.COM
I20260620 22:59:11.741535 6546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:11.727376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45472 (local address 127.2.237.1:35885)
0620 22:59:11.727960 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:11.727967 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:11.727991 (+ 24us) server_negotiation.cc:407] Connection header received
0620 22:59:11.728044 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:11.728050 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:11.728266 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:11.728416 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:11.729629 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:11.730717 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:11.731821 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:11.732376 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:11.732469 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:11.736319 (+ 3850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:11.736329 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:11.736344 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:11.736378 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:11.738990 (+ 2612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:11.739563 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:11.739570 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:11.739572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:11.739686 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:11.739993 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:11.739996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:11.739998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:11.740320 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:11.740638 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:11.740937 (+ 299us) server_negotiation.cc:299] Negotiation successful
0620 22:59:11.741207 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":328,"threads_started":1}
W20260620 22:59:11.742817 6263 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:45472
W20260620 22:59:11.743959 2996 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 6140: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260620 22:59:11.744139 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 6140
I20260620 22:59:11.759788 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 6274
Jun 20 22:59:11 dist-test-slave-f63b krb5kdc[6034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996351, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.2.237.2@KRBTEST.COM
I20260620 22:59:11.775532 6547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:11.760661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33100 (local address 127.2.237.2:44407)
0620 22:59:11.761235 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:11.761242 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:11.761268 (+ 26us) server_negotiation.cc:407] Connection header received
0620 22:59:11.761328 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:11.761335 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:11.761549 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:11.761706 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:11.762944 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:11.763941 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:11.765164 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:11.765696 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:11.765790 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:11.769833 (+ 4043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:11.769843 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:11.769859 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:11.769900 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:11.772734 (+ 2834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:11.773435 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:11.773445 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:11.773448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:11.773577 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:11.773903 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:11.773906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:11.773908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:11.774223 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:11.774449 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:11.774859 (+ 410us) server_negotiation.cc:299] Negotiation successful
0620 22:59:11.775186 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":314,"threads_started":1}
W20260620 22:59:11.776723 6397 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:33100
W20260620 22:59:11.777788 2996 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 6274: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260620 22:59:11.777961 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 6274
I20260620 22:59:11.793726 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 6407
Jun 20 22:59:11 dist-test-slave-f63b krb5kdc[6034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996351, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.2.237.3@KRBTEST.COM
I20260620 22:59:11.809316 6548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:11.794686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47524 (local address 127.2.237.3:35763)
0620 22:59:11.795248 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:11.795255 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:11.795284 (+ 29us) server_negotiation.cc:407] Connection header received
0620 22:59:11.795345 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:11.795352 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:11.795566 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:11.795693 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:11.796965 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:11.797966 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:11.799338 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:11.799909 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:11.800006 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:11.804180 (+ 4174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:11.804191 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:11.804206 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:11.804238 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:11.806950 (+ 2712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:11.807534 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:11.807541 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:11.807543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:11.807650 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:11.807945 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:11.807948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:11.807950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:11.808251 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:11.808485 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:11.808722 (+ 237us) server_negotiation.cc:299] Negotiation successful
0620 22:59:11.808933 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":258,"threads_started":1}
W20260620 22:59:11.810681 6530 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:47524
W20260620 22:59:11.811774 2996 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 6407: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260620 22:59:11.811949 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 6407
I20260620 22:59:11.827322 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 6047
I20260620 22:59:11.828486 6109 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:11.930413 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 6047
2026-06-20T22:59:11Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3635 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 22:59:12 dist-test-slave-f63b krb5kdc[6552](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 22:59:12 dist-test-slave-f63b krb5kdc[6552](info): set up 2 sockets
Jun 20 22:59:12 dist-test-slave-f63b krb5kdc[6552](info): commencing operation
krb5kdc: starting...
W20260620 22:59:13.059115 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.003s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 22:59:13 dist-test-slave-f63b krb5kdc[6552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T22:59:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T22:59:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
I20260620 22:59:13.257594 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:45469
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:45469
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:13.585835 6565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:13.586170 6565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:13.586273 6565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:13.593667 6565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 22:59:13.593798 6565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:13.593848 6565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:13.593900 6565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 22:59:13.593936 6565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 22:59:13.603926 6565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:45469
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:45469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:13.605225 6565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:13.606456 6565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:13.618301 6571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:13.618301 6570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:13.618950 6573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:13.619334 6565 server_base.cc:1034] running on GCE node
I20260620 22:59:13.620453 6565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:13.622017 6565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:13.625382 6565 hybrid_clock.cc:648] HybridClock initialized: now 1781996353625262 us; error 107 us; skew 500 ppm
I20260620 22:59:13.625798 6565 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:13 dist-test-slave-f63b krb5kdc[6552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996353, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:13.632834 6565 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 22:59:13.634469 6565 webserver.cc:460] Webserver started at http://127.2.237.62:46781/ using document root <none> and password file <none>
I20260620 22:59:13.635212 6565 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:13.635356 6565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:13.635670 6565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:13.638849 6565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "a6e18d697a834bf9a0b29d5b32cf0af9"
format_stamp: "Formatted at 2026-06-20 22:59:13 on dist-test-slave-f63b"
server_key: "b59fdca46121fa360ced9776a84e37f2"
server_key_iv: "beb8912444fb965d9638e7f6cf166fe3"
server_key_version: "encryptionkey@0"
I20260620 22:59:13.639628 6565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "a6e18d697a834bf9a0b29d5b32cf0af9"
format_stamp: "Formatted at 2026-06-20 22:59:13 on dist-test-slave-f63b"
server_key: "b59fdca46121fa360ced9776a84e37f2"
server_key_iv: "beb8912444fb965d9638e7f6cf166fe3"
server_key_version: "encryptionkey@0"
I20260620 22:59:13.644799 6565 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260620 22:59:13.648478 6580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:13.649793 6565 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 22:59:13.649976 6565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "a6e18d697a834bf9a0b29d5b32cf0af9"
format_stamp: "Formatted at 2026-06-20 22:59:13 on dist-test-slave-f63b"
server_key: "b59fdca46121fa360ced9776a84e37f2"
server_key_iv: "beb8912444fb965d9638e7f6cf166fe3"
server_key_version: "encryptionkey@0"
I20260620 22:59:13.650175 6565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:13.665441 6565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:13.669417 6565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:13.669705 6565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:13.687846 6565 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:45469
I20260620 22:59:13.687868 6631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:45469 every 8 connection(s)
I20260620 22:59:13.689433 6565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 22:59:13.693298 6632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:13.699003 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 6565
I20260620 22:59:13.699237 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 22:59:13.699846 2996 external_mini_cluster.cc:1428] Setting key 9fb5f68e4b0bd01c26c7bd5c82641dd8
I20260620 22:59:13.703804 6632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6e18d697a834bf9a0b29d5b32cf0af9: Bootstrap starting.
I20260620 22:59:13.707471 6632 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6e18d697a834bf9a0b29d5b32cf0af9: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:13.708567 6632 log.cc:824] T 00000000000000000000000000000000 P a6e18d697a834bf9a0b29d5b32cf0af9: Log is configured to *not* fsync() on all Append() calls
I20260620 22:59:13.711637 6632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6e18d697a834bf9a0b29d5b32cf0af9: No bootstrap required, opened a new log
Jun 20 22:59:13 dist-test-slave-f63b krb5kdc[6552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:59:13.719821 6632 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6e18d697a834bf9a0b29d5b32cf0af9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6e18d697a834bf9a0b29d5b32cf0af9" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 45469 } }
I20260620 22:59:13.720220 6632 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6e18d697a834bf9a0b29d5b32cf0af9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:13.720362 6632 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6e18d697a834bf9a0b29d5b32cf0af9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6e18d697a834bf9a0b29d5b32cf0af9, State: Initialized, Role: FOLLOWER
I20260620 22:59:13.721079 6632 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6e18d697a834bf9a0b29d5b32cf0af9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6e18d697a834bf9a0b29d5b32cf0af9" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 45469 } }
I20260620 22:59:13.721369 6632 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6e18d697a834bf9a0b29d5b32cf0af9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 22:59:13.721536 6632 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6e18d697a834bf9a0b29d5b32cf0af9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 22:59:13.721740 6632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6e18d697a834bf9a0b29d5b32cf0af9 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:13.723762 6635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:13.701790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58746 (local address 127.2.237.62:45469)
0620 22:59:13.702553 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:13.702565 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:13.702663 (+ 98us) server_negotiation.cc:407] Connection header received
0620 22:59:13.704382 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:13.704404 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:13.704779 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:13.705094 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:13.707003 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:13.708364 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:13.709987 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:13.710816 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:13.710962 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:13.715456 (+ 4494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:13.715466 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:13.715488 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:13.715529 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:13.719050 (+ 3521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:13.719787 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:13.719795 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:13.719802 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:13.719971 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:13.720422 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:13.720428 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:13.720434 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:13.721134 (+ 700us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:13.721444 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:13.721916 (+ 472us) server_negotiation.cc:299] Negotiation successful
0620 22:59:13.722371 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":367,"threads_started":1}
I20260620 22:59:13.725020 6632 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6e18d697a834bf9a0b29d5b32cf0af9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6e18d697a834bf9a0b29d5b32cf0af9" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 45469 } }
I20260620 22:59:13.725569 6632 leader_election.cc:302] T 00000000000000000000000000000000 P a6e18d697a834bf9a0b29d5b32cf0af9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6e18d697a834bf9a0b29d5b32cf0af9; no voters:
I20260620 22:59:13.726334 6632 leader_election.cc:288] T 00000000000000000000000000000000 P a6e18d697a834bf9a0b29d5b32cf0af9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 22:59:13.726537 6637 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6e18d697a834bf9a0b29d5b32cf0af9 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:59:13.727988 6637 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6e18d697a834bf9a0b29d5b32cf0af9 [term 1 LEADER]: Becoming Leader. State: Replica: a6e18d697a834bf9a0b29d5b32cf0af9, State: Running, Role: LEADER
I20260620 22:59:13.728574 6637 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6e18d697a834bf9a0b29d5b32cf0af9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6e18d697a834bf9a0b29d5b32cf0af9" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 45469 } }
I20260620 22:59:13.729188 6632 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6e18d697a834bf9a0b29d5b32cf0af9 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 22:59:13.732160 6638 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6e18d697a834bf9a0b29d5b32cf0af9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6e18d697a834bf9a0b29d5b32cf0af9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6e18d697a834bf9a0b29d5b32cf0af9" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 45469 } } }
I20260620 22:59:13.732976 6638 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6e18d697a834bf9a0b29d5b32cf0af9 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:13.732317 6639 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6e18d697a834bf9a0b29d5b32cf0af9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6e18d697a834bf9a0b29d5b32cf0af9. Latest consensus state: current_term: 1 leader_uuid: "a6e18d697a834bf9a0b29d5b32cf0af9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6e18d697a834bf9a0b29d5b32cf0af9" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 45469 } } }
I20260620 22:59:13.734090 6639 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6e18d697a834bf9a0b29d5b32cf0af9 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:13.735287 6647 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 22:59:13.739554 6647 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 22:59:13.748065 6647 catalog_manager.cc:1347] Generated new cluster ID: 51457e73e6d941228fb01fc9b41ca093
I20260620 22:59:13.748183 6647 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 22:59:13.765683 6647 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 22:59:13.767103 6647 catalog_manager.cc:1504] Loading token signing keys...
I20260620 22:59:13.778745 6647 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6e18d697a834bf9a0b29d5b32cf0af9: Generated new TSK 0
I20260620 22:59:13.779657 6647 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.237.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
I20260620 22:59:13.838907 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:0
--local_ip_for_outbound_sockets=127.2.237.1
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:45469
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:14.168718 6658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:14.169001 6658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:14.169155 6658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:14.176889 6658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:14.177018 6658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:14.177232 6658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 22:59:14.187394 6658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:45469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:14.188867 6658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:14.190210 6658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:14.203267 6663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:14.203353 6664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:14.203303 6666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:14.205197 6658 server_base.cc:1034] running on GCE node
I20260620 22:59:14.205824 6658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:14.206840 6658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:14.208066 6658 hybrid_clock.cc:648] HybridClock initialized: now 1781996354208025 us; error 58 us; skew 500 ppm
I20260620 22:59:14.208434 6658 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:14 dist-test-slave-f63b krb5kdc[6552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996354, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:14.211706 6658 init.cc:377] Logged in from keytab as kudu/127.2.237.1@KRBTEST.COM (short username kudu)
I20260620 22:59:14.213093 6658 webserver.cc:460] Webserver started at http://127.2.237.1:34011/ using document root <none> and password file <none>
I20260620 22:59:14.213801 6658 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:14.213896 6658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:14.214148 6658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:14.217092 6658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-0/data/instance:
uuid: "2122c2919afe427d87e3a7267570d857"
format_stamp: "Formatted at 2026-06-20 22:59:14 on dist-test-slave-f63b"
server_key: "f0aadbed2e597e092b467cf97e65b7b8"
server_key_iv: "c3d0308c310af54faff6ae5f8e591856"
server_key_version: "encryptionkey@0"
I20260620 22:59:14.217808 6658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance:
uuid: "2122c2919afe427d87e3a7267570d857"
format_stamp: "Formatted at 2026-06-20 22:59:14 on dist-test-slave-f63b"
server_key: "f0aadbed2e597e092b467cf97e65b7b8"
server_key_iv: "c3d0308c310af54faff6ae5f8e591856"
server_key_version: "encryptionkey@0"
I20260620 22:59:14.222935 6658 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260620 22:59:14.227438 6673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:14.229417 6658 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260620 22:59:14.229596 6658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "2122c2919afe427d87e3a7267570d857"
format_stamp: "Formatted at 2026-06-20 22:59:14 on dist-test-slave-f63b"
server_key: "f0aadbed2e597e092b467cf97e65b7b8"
server_key_iv: "c3d0308c310af54faff6ae5f8e591856"
server_key_version: "encryptionkey@0"
I20260620 22:59:14.229846 6658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:14.278744 6658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:14.283399 6658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:14.283720 6658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:14.284740 6658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:59:14.286118 6658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:14.286223 6658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:14.286382 6658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:14.286459 6658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:14.321621 6658 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:37425
I20260620 22:59:14.321679 6785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:37425 every 8 connection(s)
I20260620 22:59:14.323030 6658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 22:59:14.331002 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 6658
I20260620 22:59:14.331219 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance
I20260620 22:59:14.331976 2996 external_mini_cluster.cc:1428] Setting key da80f1c704735423016c56d3544f9d92
Jun 20 22:59:14 dist-test-slave-f63b krb5kdc[6552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996354, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:59:14.344786 6788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:14.325758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:54503 (local address 127.2.237.62:45469)
0620 22:59:14.326169 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:14.326173 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:14.327362 (+ 1189us) server_negotiation.cc:407] Connection header received
0620 22:59:14.329261 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:14.329266 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:14.329350 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:14.329480 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:14.331102 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:14.332027 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:14.334116 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:14.334646 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:14.334792 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:14.339141 (+ 4349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:14.339146 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:14.339150 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:14.339180 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:14.341753 (+ 2573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:14.342464 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:14.342474 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:14.342477 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:14.342647 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:14.343081 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:14.343086 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:14.343089 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:14.343350 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:14.343497 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:14.344211 (+ 714us) server_negotiation.cc:299] Negotiation successful
0620 22:59:14.344420 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":202,"threads_started":1}
I20260620 22:59:14.345767 6789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:14.326398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:45469 (local address 127.2.237.1:54503)
0620 22:59:14.327198 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:14.327237 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:14.328950 (+ 1713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:14.329728 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:14.329736 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:14.330249 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:14.330938 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:14.330955 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:14.332244 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:14.332250 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:14.333963 (+ 1713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:14.333981 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:14.334890 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:14.334898 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:14.335238 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:14.336099 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:14.336134 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:14.338908 (+ 2774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:14.341894 (+ 2986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:14.341903 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:14.341919 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:14.342337 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:14.342767 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:14.342772 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:14.342775 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:14.342963 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:14.343474 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:14.343482 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:14.343831 (+ 349us) client_negotiation.cc:769] Sending connection context
0620 22:59:14.344107 (+ 276us) client_negotiation.cc:240] Negotiation successful
0620 22:59:14.344655 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":376,"threads_started":1}
I20260620 22:59:14.347345 6786 heartbeater.cc:347] Connected to a master server at 127.2.237.62:45469
I20260620 22:59:14.347662 6786 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:14.348385 6786 heartbeater.cc:511] Master 127.2.237.62:45469 requested a full tablet report, sending...
I20260620 22:59:14.350490 6597 ts_manager.cc:194] Registered new tserver with Master: 2122c2919afe427d87e3a7267570d857 (127.2.237.1:37425)
WARNING: no policy specified for kudu/127.2.237.2@KRBTEST.COM; defaulting to no policy
I20260620 22:59:14.352013 6597 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.1@KRBTEST.COM'} at 127.2.237.1:54503
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
I20260620 22:59:14.377029 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:0
--local_ip_for_outbound_sockets=127.2.237.2
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:45469
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:14.720762 6792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:14.721062 6792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:14.721158 6792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:14.728615 6792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:14.728734 6792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:14.728854 6792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 22:59:14.739399 6792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:45469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:14.740831 6792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:14.742398 6792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:14.755686 6798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:14.755746 6800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:14.755709 6797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:14.756908 6792 server_base.cc:1034] running on GCE node
I20260620 22:59:14.757556 6792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:14.758543 6792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:14.759886 6792 hybrid_clock.cc:648] HybridClock initialized: now 1781996354759819 us; error 65 us; skew 500 ppm
I20260620 22:59:14.760241 6792 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:14 dist-test-slave-f63b krb5kdc[6552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996354, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:14.763505 6792 init.cc:377] Logged in from keytab as kudu/127.2.237.2@KRBTEST.COM (short username kudu)
I20260620 22:59:14.765025 6792 webserver.cc:460] Webserver started at http://127.2.237.2:37533/ using document root <none> and password file <none>
I20260620 22:59:14.765746 6792 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:14.765892 6792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:14.766209 6792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:14.769269 6792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-1/data/instance:
uuid: "c9bf950020304103b7d6b8f22eeefd3d"
format_stamp: "Formatted at 2026-06-20 22:59:14 on dist-test-slave-f63b"
server_key: "babdaa9817f121abd6c70354432f5e58"
server_key_iv: "a118ac1afab6d6e77f528e2dcdfaf6e9"
server_key_version: "encryptionkey@0"
I20260620 22:59:14.770063 6792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance:
uuid: "c9bf950020304103b7d6b8f22eeefd3d"
format_stamp: "Formatted at 2026-06-20 22:59:14 on dist-test-slave-f63b"
server_key: "babdaa9817f121abd6c70354432f5e58"
server_key_iv: "a118ac1afab6d6e77f528e2dcdfaf6e9"
server_key_version: "encryptionkey@0"
I20260620 22:59:14.775707 6792 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260620 22:59:14.779563 6807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:14.781059 6792 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 22:59:14.781250 6792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "c9bf950020304103b7d6b8f22eeefd3d"
format_stamp: "Formatted at 2026-06-20 22:59:14 on dist-test-slave-f63b"
server_key: "babdaa9817f121abd6c70354432f5e58"
server_key_iv: "a118ac1afab6d6e77f528e2dcdfaf6e9"
server_key_version: "encryptionkey@0"
I20260620 22:59:14.781477 6792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:14.810675 6792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:14.815399 6792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:14.815701 6792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:14.816686 6792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:59:14.818063 6792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:14.818166 6792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:14.818315 6792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:14.818387 6792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:14.851549 6792 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:37649
I20260620 22:59:14.851596 6919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:37649 every 8 connection(s)
I20260620 22:59:14.852923 6792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 22:59:14.857277 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 6792
I20260620 22:59:14.857493 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance
I20260620 22:59:14.857951 2996 external_mini_cluster.cc:1428] Setting key 909780b23ddb0b81fced297e69057472
Jun 20 22:59:14 dist-test-slave-f63b krb5kdc[6552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996354, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:59:14.874739 6922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:14.855658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:60735 (local address 127.2.237.62:45469)
0620 22:59:14.856108 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:14.856113 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:14.857134 (+ 1021us) server_negotiation.cc:407] Connection header received
0620 22:59:14.859214 (+ 2080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:14.859220 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:14.859308 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:14.859433 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:14.861030 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:14.861759 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:14.863398 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:14.863882 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:14.864034 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:14.868255 (+ 4221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:14.868260 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:14.868263 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:14.868298 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:14.871115 (+ 2817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:14.872220 (+ 1105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:14.872227 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:14.872235 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:14.872415 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:14.872873 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:14.872878 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:14.872882 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:14.873165 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:14.873318 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:14.874061 (+ 743us) server_negotiation.cc:299] Negotiation successful
0620 22:59:14.874294 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":349,"thread_start_us":245,"threads_started":1}
WARNING: no policy specified for kudu/127.2.237.3@KRBTEST.COM; defaulting to no policy
I20260620 22:59:14.875834 6923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:14.856247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:45469 (local address 127.2.237.2:60735)
0620 22:59:14.856970 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:14.857014 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:14.858911 (+ 1897us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:14.859702 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:14.859715 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:14.860290 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:14.860865 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:14.860884 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:14.861910 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:14.861916 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:14.863247 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:14.863259 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:14.864125 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:14.864134 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:14.864369 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:14.865087 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:14.865120 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:14.867963 (+ 2843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:14.871393 (+ 3430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:14.871403 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:14.871456 (+ 53us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:14.872077 (+ 621us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:14.872531 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:14.872537 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:14.872541 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:14.872749 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:14.873294 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:14.873302 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:14.873690 (+ 388us) client_negotiation.cc:769] Sending connection context
0620 22:59:14.873960 (+ 270us) client_negotiation.cc:240] Negotiation successful
0620 22:59:14.874534 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":292,"threads_started":1}
I20260620 22:59:14.877434 6920 heartbeater.cc:347] Connected to a master server at 127.2.237.62:45469
I20260620 22:59:14.877772 6920 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.3@KRBTEST.COM" created.
I20260620 22:59:14.878625 6920 heartbeater.cc:511] Master 127.2.237.62:45469 requested a full tablet report, sending...
I20260620 22:59:14.880177 6597 ts_manager.cc:194] Registered new tserver with Master: c9bf950020304103b7d6b8f22eeefd3d (127.2.237.2:37649)
I20260620 22:59:14.881006 6597 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.2@KRBTEST.COM'} at 127.2.237.2:60735
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
I20260620 22:59:14.901948 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:0
--local_ip_for_outbound_sockets=127.2.237.3
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:45469
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:46319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:15.232327 6926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:15.232663 6926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:15.232811 6926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:15.240513 6926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:15.240630 6926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:15.240813 6926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 22:59:15.250841 6926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:46319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:45469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:15.252358 6926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:15.254236 6926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:15.268573 6931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:15.268549 6932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:15.268517 6934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:15.269110 6926 server_base.cc:1034] running on GCE node
I20260620 22:59:15.269917 6926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:15.271054 6926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:15.272302 6926 hybrid_clock.cc:648] HybridClock initialized: now 1781996355272242 us; error 47 us; skew 500 ppm
I20260620 22:59:15.272639 6926 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:15 dist-test-slave-f63b krb5kdc[6552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996355, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:15.276023 6926 init.cc:377] Logged in from keytab as kudu/127.2.237.3@KRBTEST.COM (short username kudu)
I20260620 22:59:15.277473 6926 webserver.cc:460] Webserver started at http://127.2.237.3:33393/ using document root <none> and password file <none>
I20260620 22:59:15.278142 6926 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:15.278254 6926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:15.278499 6926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:15.281577 6926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-2/data/instance:
uuid: "ad7d07817f054b60a2d055fcbc14f0da"
format_stamp: "Formatted at 2026-06-20 22:59:15 on dist-test-slave-f63b"
server_key: "8c227a7885dc7ca05bc38432bb9f8798"
server_key_iv: "0d46aec38f50f65166c05f0c6ec48b30"
server_key_version: "encryptionkey@0"
I20260620 22:59:15.282375 6926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance:
uuid: "ad7d07817f054b60a2d055fcbc14f0da"
format_stamp: "Formatted at 2026-06-20 22:59:15 on dist-test-slave-f63b"
server_key: "8c227a7885dc7ca05bc38432bb9f8798"
server_key_iv: "0d46aec38f50f65166c05f0c6ec48b30"
server_key_version: "encryptionkey@0"
I20260620 22:59:15.288281 6926 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260620 22:59:15.292027 6941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:15.293457 6926 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 22:59:15.293622 6926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "ad7d07817f054b60a2d055fcbc14f0da"
format_stamp: "Formatted at 2026-06-20 22:59:15 on dist-test-slave-f63b"
server_key: "8c227a7885dc7ca05bc38432bb9f8798"
server_key_iv: "0d46aec38f50f65166c05f0c6ec48b30"
server_key_version: "encryptionkey@0"
I20260620 22:59:15.293841 6926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:15.307649 6926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:15.312378 6926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:15.312709 6926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:15.313723 6926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:59:15.315249 6926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:15.315323 6926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:15.315457 6926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:15.315510 6926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:15.349283 6926 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:39643
I20260620 22:59:15.349323 7053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:39643 every 8 connection(s)
I20260620 22:59:15.350631 6926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 22:59:15.351886 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 6926
I20260620 22:59:15.352077 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance
I20260620 22:59:15.352609 2996 external_mini_cluster.cc:1428] Setting key a6085052aff6568a71e9ae1891b5adb2
I20260620 22:59:15.355369 6786 heartbeater.cc:503] Master 127.2.237.62:45469 was elected leader, sending a full tablet report...
Jun 20 22:59:15 dist-test-slave-f63b krb5kdc[6552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996355, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:59:15.371685 6922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:15.353724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:52597 (local address 127.2.237.62:45469)
0620 22:59:15.353942 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:15.353946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:15.355216 (+ 1270us) server_negotiation.cc:407] Connection header received
0620 22:59:15.357813 (+ 2597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:15.357817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:15.357878 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:15.357962 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:15.359591 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:15.360312 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:15.361910 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:15.362323 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:15.362422 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:15.365875 (+ 3453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:15.365880 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:15.365883 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:15.365916 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:15.368863 (+ 2947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:15.369560 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:15.369563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:15.369566 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:15.369743 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:15.370138 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:15.370141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:15.370144 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:15.370413 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:15.370522 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:15.371254 (+ 732us) server_negotiation.cc:299] Negotiation successful
0620 22:59:15.371421 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 22:59:15.372748 7056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:15.354212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:45469 (local address 127.2.237.3:52597)
0620 22:59:15.355054 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:15.355115 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:15.357476 (+ 2361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:15.358199 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:15.358208 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:15.358864 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:15.359443 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:15.359461 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:15.360460 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:15.360464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:15.361775 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:15.361787 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:15.362511 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:15.362515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:15.362808 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:15.363431 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:15.363452 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:15.365609 (+ 2157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:15.368992 (+ 3383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:15.368999 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:15.369014 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:15.369453 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:15.369837 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:15.369840 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:15.369842 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:15.370026 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:15.370516 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:15.370522 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:15.370912 (+ 390us) client_negotiation.cc:769] Sending connection context
0620 22:59:15.371133 (+ 221us) client_negotiation.cc:240] Negotiation successful
0620 22:59:15.371527 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":408,"threads_started":1}
I20260620 22:59:15.374146 7054 heartbeater.cc:347] Connected to a master server at 127.2.237.62:45469
I20260620 22:59:15.374468 7054 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:15.375237 7054 heartbeater.cc:511] Master 127.2.237.62:45469 requested a full tablet report, sending...
I20260620 22:59:15.376646 6597 ts_manager.cc:194] Registered new tserver with Master: ad7d07817f054b60a2d055fcbc14f0da (127.2.237.3:39643)
I20260620 22:59:15.377460 6597 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.3@KRBTEST.COM'} at 127.2.237.3:52597
I20260620 22:59:15.380963 2996 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 22:59:15 dist-test-slave-f63b krb5kdc[6552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 22:59:15 dist-test-slave-f63b krb5kdc[6552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.1@KRBTEST.COM
I20260620 22:59:15.429147 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:15.413726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39214 (local address 127.2.237.1:37425)
0620 22:59:15.414390 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:15.414396 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:15.414421 (+ 25us) server_negotiation.cc:407] Connection header received
0620 22:59:15.414481 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:15.414487 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:15.414776 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:15.414946 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:15.416526 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:15.417523 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:15.418828 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:15.419401 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:15.419537 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:15.423427 (+ 3890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:15.423438 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:15.423454 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:15.423492 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:15.426233 (+ 2741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:15.426972 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:15.426979 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:15.426981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:15.427097 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:15.427481 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:15.427485 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:15.427487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:15.427819 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:15.428143 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:15.428440 (+ 297us) server_negotiation.cc:299] Negotiation successful
0620 22:59:15.428713 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":341,"threads_started":1}
Jun 20 22:59:15 dist-test-slave-f63b krb5kdc[6552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:59:15.447467 6922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:15.433581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58748 (local address 127.2.237.62:45469)
0620 22:59:15.433842 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:15.433849 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:15.434066 (+ 217us) server_negotiation.cc:407] Connection header received
0620 22:59:15.434224 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:15.434228 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:15.434298 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:15.434413 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:15.435618 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:15.436367 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:15.437544 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:15.437955 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:15.438049 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:15.441956 (+ 3907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:15.441961 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:15.441965 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:15.441998 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:15.444738 (+ 2740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:15.445423 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:15.445427 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:15.445430 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:15.445559 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:15.445950 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:15.445953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:15.445955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:15.446262 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:15.446451 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:15.446891 (+ 440us) server_negotiation.cc:299] Negotiation successful
0620 22:59:15.447086 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
W20260620 22:59:15.448168 6597 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58748
I20260620 22:59:15.449364 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 6658
I20260620 22:59:15.461796 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:15.450166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39228 (local address 127.2.237.1:37425)
0620 22:59:15.450343 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:15.450347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:15.450638 (+ 291us) server_negotiation.cc:407] Connection header received
0620 22:59:15.450851 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:15.450854 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:15.450934 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:15.451016 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:15.452199 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:15.452898 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:15.454152 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:15.454553 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:15.454706 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:15.456925 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:15.456929 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:15.456933 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:15.456963 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:15.459552 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:15.460222 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:15.460225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:15.460227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:15.460351 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:15.460677 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:15.460680 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:15.460682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:15.460902 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:15.461075 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:15.461351 (+ 276us) server_negotiation.cc:299] Negotiation successful
0620 22:59:15.461501 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 22:59:15.462497 6781 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:15.544685 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 6658
I20260620 22:59:15.561198 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 6792
Jun 20 22:59:15 dist-test-slave-f63b krb5kdc[6552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.2@KRBTEST.COM
I20260620 22:59:15.577320 7066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:15.562119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55684 (local address 127.2.237.2:37649)
0620 22:59:15.562721 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:15.562728 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:15.562754 (+ 26us) server_negotiation.cc:407] Connection header received
0620 22:59:15.562813 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:15.562820 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:15.563034 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:15.563209 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:15.564453 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:15.565440 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:15.566756 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:15.567352 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:15.567494 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:15.571717 (+ 4223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:15.571736 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:15.571750 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:15.571785 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:15.574535 (+ 2750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:15.575233 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:15.575241 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:15.575243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:15.575364 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:15.575751 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:15.575760 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:15.575763 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:15.576085 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:15.576389 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:15.576717 (+ 328us) server_negotiation.cc:299] Negotiation successful
0620 22:59:15.576982 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":236,"threads_started":1}
I20260620 22:59:15.578816 6915 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:15.661484 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 6792
I20260620 22:59:15.677238 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 6926
Jun 20 22:59:15 dist-test-slave-f63b krb5kdc[6552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.3@KRBTEST.COM
I20260620 22:59:15.695003 7068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:15.678201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37780 (local address 127.2.237.3:39643)
0620 22:59:15.678887 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:15.678894 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:15.678921 (+ 27us) server_negotiation.cc:407] Connection header received
0620 22:59:15.678978 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:15.678984 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:15.679211 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:15.679386 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:15.680614 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:15.681649 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:15.683011 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:15.683594 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:15.683742 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:15.688204 (+ 4462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:15.688218 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:15.688239 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:15.688280 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:15.691990 (+ 3710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:15.692762 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:15.692772 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:15.692776 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:15.692915 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:15.693289 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:15.693293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:15.693297 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:15.693731 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:15.694033 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:15.694313 (+ 280us) server_negotiation.cc:299] Negotiation successful
0620 22:59:15.694595 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":324,"threads_started":1}
I20260620 22:59:15.696477 7049 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:15.780151 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 6926
I20260620 22:59:15.795826 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 6565
I20260620 22:59:15.796979 6627 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:15.898837 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 6565
2026-06-20T22:59:15Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3970 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 22:59:15 dist-test-slave-f63b krb5kdc[7073](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 22:59:15 dist-test-slave-f63b krb5kdc[7073](info): set up 2 sockets
Jun 20 22:59:15 dist-test-slave-f63b krb5kdc[7073](info): commencing operation
krb5kdc: starting...
W20260620 22:59:16.991128 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.006s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 22:59:17 dist-test-slave-f63b krb5kdc[7073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T22:59:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T22:59:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
I20260620 22:59:17.186298 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:45559
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:42671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:45559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:17.505533 7086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:17.505843 7086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:17.505935 7086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:17.513211 7086 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 22:59:17.513326 7086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:17.513404 7086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:17.513495 7086 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 22:59:17.513562 7086 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 22:59:17.523175 7086 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:42671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:45559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:45559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:17.524466 7086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:17.525693 7086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:17.537591 7094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:17.537631 7092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:17.537562 7091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:17.538295 7086 server_base.cc:1034] running on GCE node
I20260620 22:59:17.539309 7086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:17.540848 7086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:17.542111 7086 hybrid_clock.cc:648] HybridClock initialized: now 1781996357542042 us; error 48 us; skew 500 ppm
I20260620 22:59:17.542492 7086 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:17 dist-test-slave-f63b krb5kdc[7073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996357, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:17.545723 7086 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 22:59:17.547012 7086 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:17.547164 7086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:17.547503 7086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:17.550674 7086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "5b46bf13296d4fa4aec640a8ff6843a6"
format_stamp: "Formatted at 2026-06-20 22:59:17 on dist-test-slave-f63b"
server_key: "6e2edd2df45627ae61fdeded7e405f2a"
server_key_iv: "f2a530efacca3cbb8d913a930c9fc33f"
server_key_version: "encryptionkey@0"
I20260620 22:59:17.551532 7086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "5b46bf13296d4fa4aec640a8ff6843a6"
format_stamp: "Formatted at 2026-06-20 22:59:17 on dist-test-slave-f63b"
server_key: "6e2edd2df45627ae61fdeded7e405f2a"
server_key_iv: "f2a530efacca3cbb8d913a930c9fc33f"
server_key_version: "encryptionkey@0"
I20260620 22:59:17.556823 7086 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260620 22:59:17.560338 7100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:17.561681 7086 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 22:59:17.561897 7086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "5b46bf13296d4fa4aec640a8ff6843a6"
format_stamp: "Formatted at 2026-06-20 22:59:17 on dist-test-slave-f63b"
server_key: "6e2edd2df45627ae61fdeded7e405f2a"
server_key_iv: "f2a530efacca3cbb8d913a930c9fc33f"
server_key_version: "encryptionkey@0"
I20260620 22:59:17.562155 7086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:17.581488 7086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:17.585930 7086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:17.586251 7086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:17.605883 7086 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:45559
I20260620 22:59:17.605896 7151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:45559 every 8 connection(s)
I20260620 22:59:17.607034 7086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 22:59:17.610998 7152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:17.615507 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 7086
I20260620 22:59:17.615772 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 22:59:17.616362 2996 external_mini_cluster.cc:1428] Setting key 4404f707de7c0d844bd7c7c7546a7500
I20260620 22:59:17.623085 7152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b46bf13296d4fa4aec640a8ff6843a6: Bootstrap starting.
I20260620 22:59:17.626856 7152 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b46bf13296d4fa4aec640a8ff6843a6: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:17.627879 7152 log.cc:824] T 00000000000000000000000000000000 P 5b46bf13296d4fa4aec640a8ff6843a6: Log is configured to *not* fsync() on all Append() calls
I20260620 22:59:17.630967 7152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b46bf13296d4fa4aec640a8ff6843a6: No bootstrap required, opened a new log
Jun 20 22:59:17 dist-test-slave-f63b krb5kdc[7073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:59:17.639504 7152 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b46bf13296d4fa4aec640a8ff6843a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b46bf13296d4fa4aec640a8ff6843a6" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 45559 } }
I20260620 22:59:17.639932 7152 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b46bf13296d4fa4aec640a8ff6843a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:17.640071 7152 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b46bf13296d4fa4aec640a8ff6843a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b46bf13296d4fa4aec640a8ff6843a6, State: Initialized, Role: FOLLOWER
I20260620 22:59:17.640797 7152 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b46bf13296d4fa4aec640a8ff6843a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b46bf13296d4fa4aec640a8ff6843a6" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 45559 } }
I20260620 22:59:17.641072 7152 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b46bf13296d4fa4aec640a8ff6843a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 22:59:17.641238 7152 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b46bf13296d4fa4aec640a8ff6843a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 22:59:17.641440 7152 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b46bf13296d4fa4aec640a8ff6843a6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:17.642460 7155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:17.618531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36674 (local address 127.2.237.62:45559)
0620 22:59:17.619470 (+ 939us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:17.619486 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:17.619543 (+ 57us) server_negotiation.cc:407] Connection header received
0620 22:59:17.621941 (+ 2398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:17.621969 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:17.622482 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:17.622955 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:17.624361 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:17.626447 (+ 2086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:17.628174 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:17.629011 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:17.629262 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:17.634100 (+ 4838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:17.634112 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:17.634129 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:17.634178 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:17.637757 (+ 3579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:17.638613 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:17.638624 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:17.638634 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:17.638825 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:17.639255 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:17.639262 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:17.639266 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:17.639889 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:17.640224 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:17.640709 (+ 485us) server_negotiation.cc:299] Negotiation successful
0620 22:59:17.641161 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":626,"thread_start_us":374,"threads_started":1}
I20260620 22:59:17.644007 7152 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b46bf13296d4fa4aec640a8ff6843a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b46bf13296d4fa4aec640a8ff6843a6" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 45559 } }
I20260620 22:59:17.644502 7152 leader_election.cc:302] T 00000000000000000000000000000000 P 5b46bf13296d4fa4aec640a8ff6843a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b46bf13296d4fa4aec640a8ff6843a6; no voters:
I20260620 22:59:17.645179 7152 leader_election.cc:288] T 00000000000000000000000000000000 P 5b46bf13296d4fa4aec640a8ff6843a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 22:59:17.645292 7157 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b46bf13296d4fa4aec640a8ff6843a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:59:17.646124 7157 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b46bf13296d4fa4aec640a8ff6843a6 [term 1 LEADER]: Becoming Leader. State: Replica: 5b46bf13296d4fa4aec640a8ff6843a6, State: Running, Role: LEADER
I20260620 22:59:17.646665 7157 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b46bf13296d4fa4aec640a8ff6843a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b46bf13296d4fa4aec640a8ff6843a6" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 45559 } }
I20260620 22:59:17.647665 7152 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b46bf13296d4fa4aec640a8ff6843a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 22:59:17.649456 7158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b46bf13296d4fa4aec640a8ff6843a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b46bf13296d4fa4aec640a8ff6843a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b46bf13296d4fa4aec640a8ff6843a6" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 45559 } } }
I20260620 22:59:17.649850 7158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b46bf13296d4fa4aec640a8ff6843a6 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:17.653725 7159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b46bf13296d4fa4aec640a8ff6843a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b46bf13296d4fa4aec640a8ff6843a6. Latest consensus state: current_term: 1 leader_uuid: "5b46bf13296d4fa4aec640a8ff6843a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b46bf13296d4fa4aec640a8ff6843a6" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 45559 } } }
I20260620 22:59:17.654172 7159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b46bf13296d4fa4aec640a8ff6843a6 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:17.655669 7162 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 22:59:17.661397 7162 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 22:59:17.670406 7162 catalog_manager.cc:1347] Generated new cluster ID: 2d63036a7ae54010bacc74055faac4be
I20260620 22:59:17.670548 7162 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 22:59:17.697934 7162 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 22:59:17.699608 7162 catalog_manager.cc:1504] Loading token signing keys...
I20260620 22:59:17.710127 7162 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b46bf13296d4fa4aec640a8ff6843a6: Generated new TSK 0
I20260620 22:59:17.711297 7162 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.237.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
I20260620 22:59:17.771941 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:0
--local_ip_for_outbound_sockets=127.2.237.1
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:45559
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:42671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:18.103351 7178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:18.103643 7178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:18.103857 7178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:18.111270 7178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:18.111398 7178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:18.111591 7178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 22:59:18.121469 7178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:42671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:45559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:18.122921 7178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:18.124244 7178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:18.137339 7183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:18.137241 7186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:18.137269 7184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:18.138121 7178 server_base.cc:1034] running on GCE node
I20260620 22:59:18.138880 7178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:18.139822 7178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:18.141050 7178 hybrid_clock.cc:648] HybridClock initialized: now 1781996358140988 us; error 46 us; skew 500 ppm
I20260620 22:59:18.141388 7178 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:18 dist-test-slave-f63b krb5kdc[7073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996358, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:18.144881 7178 init.cc:377] Logged in from keytab as kudu/127.2.237.1@KRBTEST.COM (short username kudu)
I20260620 22:59:18.146109 7178 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:18.146251 7178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:18.146620 7178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:18.149709 7178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-0/data/instance:
uuid: "85315db07a5e48d3be97b93b338c1d2e"
format_stamp: "Formatted at 2026-06-20 22:59:18 on dist-test-slave-f63b"
server_key: "4b2d853155ee7c59b2cc2db45f5910d2"
server_key_iv: "2a47026dd6870a98f3528fd92ed4988e"
server_key_version: "encryptionkey@0"
I20260620 22:59:18.150519 7178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance:
uuid: "85315db07a5e48d3be97b93b338c1d2e"
format_stamp: "Formatted at 2026-06-20 22:59:18 on dist-test-slave-f63b"
server_key: "4b2d853155ee7c59b2cc2db45f5910d2"
server_key_iv: "2a47026dd6870a98f3528fd92ed4988e"
server_key_version: "encryptionkey@0"
I20260620 22:59:18.156314 7178 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260620 22:59:18.159984 7192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:18.161427 7178 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 22:59:18.161634 7178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "85315db07a5e48d3be97b93b338c1d2e"
format_stamp: "Formatted at 2026-06-20 22:59:18 on dist-test-slave-f63b"
server_key: "4b2d853155ee7c59b2cc2db45f5910d2"
server_key_iv: "2a47026dd6870a98f3528fd92ed4988e"
server_key_version: "encryptionkey@0"
I20260620 22:59:18.161886 7178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:18.178051 7178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:18.182052 7178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:18.182453 7178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:18.183444 7178 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:59:18.184787 7178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:18.184909 7178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:18.185034 7178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:18.185129 7178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:18.218122 7178 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:41575
I20260620 22:59:18.218173 7304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:41575 every 8 connection(s)
I20260620 22:59:18.219136 7178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 22:59:18.223515 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 7178
I20260620 22:59:18.223743 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance
I20260620 22:59:18.224367 2996 external_mini_cluster.cc:1428] Setting key 6107af1b7fc4567398e6079e75733af8
Jun 20 22:59:18 dist-test-slave-f63b krb5kdc[7073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996358, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:59:18.241328 7307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:18.221623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:57545 (local address 127.2.237.62:45559)
0620 22:59:18.222053 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:18.222059 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:18.223178 (+ 1119us) server_negotiation.cc:407] Connection header received
0620 22:59:18.225100 (+ 1922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:18.225106 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:18.225212 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:18.225332 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:18.226929 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:18.227641 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:18.229222 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:18.229615 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:18.229753 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:18.234006 (+ 4253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:18.234015 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:18.234023 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:18.234071 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:18.237724 (+ 3653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:18.238550 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:18.238558 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:18.238565 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:18.238744 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:18.239198 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:18.239205 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:18.239211 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:18.239614 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:18.239793 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:18.240564 (+ 771us) server_negotiation.cc:299] Negotiation successful
0620 22:59:18.240847 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":204,"threads_started":1}
I20260620 22:59:18.242131 7308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:18.222259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:45559 (local address 127.2.237.1:57545)
0620 22:59:18.222999 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:18.223054 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:18.224826 (+ 1772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:18.225580 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:18.225589 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:18.226100 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:18.226761 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:18.226781 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:18.227804 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:18.227815 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:18.229063 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:18.229075 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:18.229846 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:18.229855 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:18.230086 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:18.230940 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:18.230980 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:18.233759 (+ 2779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:18.237892 (+ 4133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:18.237900 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:18.237914 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:18.238395 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:18.238880 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:18.238884 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:18.238887 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:18.239067 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:18.239778 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:18.239787 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:18.240142 (+ 355us) client_negotiation.cc:769] Sending connection context
0620 22:59:18.240374 (+ 232us) client_negotiation.cc:240] Negotiation successful
0620 22:59:18.240788 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":364,"threads_started":1}
WARNING: no policy specified for kudu/127.2.237.2@KRBTEST.COM; defaulting to no policy
I20260620 22:59:18.243836 7305 heartbeater.cc:347] Connected to a master server at 127.2.237.62:45559
I20260620 22:59:18.244140 7305 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:18.244822 7305 heartbeater.cc:511] Master 127.2.237.62:45559 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.2@KRBTEST.COM" created.
I20260620 22:59:18.247098 7117 ts_manager.cc:194] Registered new tserver with Master: 85315db07a5e48d3be97b93b338c1d2e (127.2.237.1:41575)
I20260620 22:59:18.249337 7117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.1@KRBTEST.COM'} at 127.2.237.1:57545
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
I20260620 22:59:18.271597 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:0
--local_ip_for_outbound_sockets=127.2.237.2
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:45559
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:42671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:18.591325 7311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:18.591611 7311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:18.591709 7311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:18.598858 7311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:18.598976 7311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:18.599102 7311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 22:59:18.608987 7311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:42671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:45559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:18.610280 7311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:18.611644 7311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:18.624334 7317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:18.624394 7316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:18.624372 7319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:18.625152 7311 server_base.cc:1034] running on GCE node
I20260620 22:59:18.625757 7311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:18.626753 7311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:18.628000 7311 hybrid_clock.cc:648] HybridClock initialized: now 1781996358627953 us; error 58 us; skew 500 ppm
I20260620 22:59:18.628333 7311 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:18 dist-test-slave-f63b krb5kdc[7073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996358, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:18.631763 7311 init.cc:377] Logged in from keytab as kudu/127.2.237.2@KRBTEST.COM (short username kudu)
I20260620 22:59:18.632920 7311 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:18.633054 7311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:18.633375 7311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:18.636441 7311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-1/data/instance:
uuid: "3e567c8f4df94104b11eb64159c666af"
format_stamp: "Formatted at 2026-06-20 22:59:18 on dist-test-slave-f63b"
server_key: "02bfb122a6796f04c72f6f131ecc054c"
server_key_iv: "dd2ab4a58ad89356442999192257a0a2"
server_key_version: "encryptionkey@0"
I20260620 22:59:18.637259 7311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance:
uuid: "3e567c8f4df94104b11eb64159c666af"
format_stamp: "Formatted at 2026-06-20 22:59:18 on dist-test-slave-f63b"
server_key: "02bfb122a6796f04c72f6f131ecc054c"
server_key_iv: "dd2ab4a58ad89356442999192257a0a2"
server_key_version: "encryptionkey@0"
I20260620 22:59:18.642391 7311 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260620 22:59:18.645815 7325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:18.647126 7311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 22:59:18.647296 7311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "3e567c8f4df94104b11eb64159c666af"
format_stamp: "Formatted at 2026-06-20 22:59:18 on dist-test-slave-f63b"
server_key: "02bfb122a6796f04c72f6f131ecc054c"
server_key_iv: "dd2ab4a58ad89356442999192257a0a2"
server_key_version: "encryptionkey@0"
I20260620 22:59:18.647548 7311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:18.684345 7311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:18.688509 7311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:18.688802 7311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:18.689754 7311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:59:18.691093 7311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:18.691217 7311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:18.691354 7311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:18.691453 7311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:18.728180 7311 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:37115
I20260620 22:59:18.728271 7437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:37115 every 8 connection(s)
I20260620 22:59:18.729110 7311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 22:59:18.729544 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 7311
I20260620 22:59:18.729822 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance
I20260620 22:59:18.730362 2996 external_mini_cluster.cc:1428] Setting key 28959b088c53452eed05453934e62f66
Jun 20 22:59:18 dist-test-slave-f63b krb5kdc[7073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996358, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.237.3@KRBTEST.COM; defaulting to no policy
I20260620 22:59:18.751120 7307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:18.731849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:58833 (local address 127.2.237.62:45559)
0620 22:59:18.732060 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:18.732066 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:18.733445 (+ 1379us) server_negotiation.cc:407] Connection header received
0620 22:59:18.735350 (+ 1905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:18.735358 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:18.735442 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:18.735568 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:18.737292 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:18.737984 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:18.739845 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:18.740292 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:18.740395 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:18.744824 (+ 4429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:18.744828 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:18.744832 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:18.744861 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:18.747456 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:18.748489 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:18.748495 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:18.748498 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:18.748655 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:18.749198 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:18.749203 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:18.749206 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:18.749464 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:18.749577 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:18.750537 (+ 960us) server_negotiation.cc:299] Negotiation successful
0620 22:59:18.750786 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260620 22:59:18.752307 7440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:18.732531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:45559 (local address 127.2.237.2:58833)
0620 22:59:18.733304 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:18.733340 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:18.735013 (+ 1673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:18.735839 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:18.735852 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:18.736544 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:18.737122 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:18.737140 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:18.738139 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:18.738144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:18.739700 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:18.739717 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:18.740514 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:18.740523 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:18.740858 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:18.741566 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:18.741601 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:18.744592 (+ 2991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:18.747592 (+ 3000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:18.747603 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:18.747624 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:18.748331 (+ 707us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:18.748813 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:18.748820 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:18.748825 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:18.749050 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:18.749604 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:18.749614 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:18.750097 (+ 483us) client_negotiation.cc:769] Sending connection context
0620 22:59:18.750445 (+ 348us) client_negotiation.cc:240] Negotiation successful
0620 22:59:18.751042 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":336,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.3@KRBTEST.COM" created.
I20260620 22:59:18.753793 7438 heartbeater.cc:347] Connected to a master server at 127.2.237.62:45559
I20260620 22:59:18.754138 7438 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:18.754961 7438 heartbeater.cc:511] Master 127.2.237.62:45559 requested a full tablet report, sending...
I20260620 22:59:18.756567 7117 ts_manager.cc:194] Registered new tserver with Master: 3e567c8f4df94104b11eb64159c666af (127.2.237.2:37115)
I20260620 22:59:18.757369 7117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.2@KRBTEST.COM'} at 127.2.237.2:58833
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
I20260620 22:59:18.777777 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:0
--local_ip_for_outbound_sockets=127.2.237.3
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:45559
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:42671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:19.106397 7443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:19.106701 7443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:19.106918 7443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:19.114143 7443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:19.114245 7443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:19.114439 7443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 22:59:19.124365 7443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:42671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:45559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:19.126188 7443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:19.127491 7443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:19.140515 7451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:19.140609 7448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:19.140592 7449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:19.144088 7443 server_base.cc:1034] running on GCE node
I20260620 22:59:19.144699 7443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:19.145670 7443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:19.146888 7443 hybrid_clock.cc:648] HybridClock initialized: now 1781996359146796 us; error 79 us; skew 500 ppm
I20260620 22:59:19.147207 7443 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:19 dist-test-slave-f63b krb5kdc[7073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996359, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:19.150275 7443 init.cc:377] Logged in from keytab as kudu/127.2.237.3@KRBTEST.COM (short username kudu)
I20260620 22:59:19.151578 7443 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:19.151715 7443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:19.152041 7443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:19.155033 7443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-2/data/instance:
uuid: "8f720e2177f941b8a9c94098861f124f"
format_stamp: "Formatted at 2026-06-20 22:59:19 on dist-test-slave-f63b"
server_key: "4dfe20d4d78fd9e8452c454045da9c24"
server_key_iv: "5bd559cee8f2bcc6a25070253ce20d2c"
server_key_version: "encryptionkey@0"
I20260620 22:59:19.155889 7443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance:
uuid: "8f720e2177f941b8a9c94098861f124f"
format_stamp: "Formatted at 2026-06-20 22:59:19 on dist-test-slave-f63b"
server_key: "4dfe20d4d78fd9e8452c454045da9c24"
server_key_iv: "5bd559cee8f2bcc6a25070253ce20d2c"
server_key_version: "encryptionkey@0"
I20260620 22:59:19.161334 7443 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260620 22:59:19.164863 7457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:19.166483 7443 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260620 22:59:19.166713 7443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "8f720e2177f941b8a9c94098861f124f"
format_stamp: "Formatted at 2026-06-20 22:59:19 on dist-test-slave-f63b"
server_key: "4dfe20d4d78fd9e8452c454045da9c24"
server_key_iv: "5bd559cee8f2bcc6a25070253ce20d2c"
server_key_version: "encryptionkey@0"
I20260620 22:59:19.166947 7443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:19.200557 7443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:19.204571 7443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:19.204877 7443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:19.205905 7443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:59:19.207252 7443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:19.207379 7443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:19.207520 7443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:19.207607 7443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:19.245467 7443 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:43845
I20260620 22:59:19.245510 7569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:43845 every 8 connection(s)
I20260620 22:59:19.246332 7443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 22:59:19.248436 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 7443
I20260620 22:59:19.248649 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance
I20260620 22:59:19.249109 2996 external_mini_cluster.cc:1428] Setting key 67d40afefda5f3c26f066f6a6ff0b60e
I20260620 22:59:19.253113 7305 heartbeater.cc:503] Master 127.2.237.62:45559 was elected leader, sending a full tablet report...
Jun 20 22:59:19 dist-test-slave-f63b krb5kdc[7073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996359, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:59:19.267280 7307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:19.249978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:60387 (local address 127.2.237.62:45559)
0620 22:59:19.250136 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:19.250140 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:19.250422 (+ 282us) server_negotiation.cc:407] Connection header received
0620 22:59:19.252852 (+ 2430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:19.252857 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:19.253000 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:19.253115 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:19.254915 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.255862 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.257378 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.257781 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.257910 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:19.261217 (+ 3307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:19.261223 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:19.261227 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:19.261266 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:19.264235 (+ 2969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:19.265045 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:19.265048 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:19.265050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:19.265210 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:19.265630 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:19.265636 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:19.265639 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:19.265918 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:19.266025 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:19.266894 (+ 869us) server_negotiation.cc:299] Negotiation successful
0620 22:59:19.267047 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 22:59:19.268337 7572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:19.249507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:45559 (local address 127.2.237.3:60387)
0620 22:59:19.250250 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:19.250297 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:19.252605 (+ 2308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:19.253334 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:19.253342 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:19.253951 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:19.254757 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:19.254775 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.256002 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.256008 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:19.257240 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:19.257251 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.258005 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.258011 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:19.258241 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:19.258975 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:19.258996 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:19.261025 (+ 2029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:19.264433 (+ 3408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:19.264441 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:19.264456 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:19.264932 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:19.265306 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:19.265310 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:19.265312 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:19.265484 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:19.266055 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:19.266061 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:19.266444 (+ 383us) client_negotiation.cc:769] Sending connection context
0620 22:59:19.266768 (+ 324us) client_negotiation.cc:240] Negotiation successful
0620 22:59:19.267135 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":557,"threads_started":1}
I20260620 22:59:19.270097 7570 heartbeater.cc:347] Connected to a master server at 127.2.237.62:45559
I20260620 22:59:19.270463 7570 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:19.271242 7570 heartbeater.cc:511] Master 127.2.237.62:45559 requested a full tablet report, sending...
I20260620 22:59:19.272523 7117 ts_manager.cc:194] Registered new tserver with Master: 8f720e2177f941b8a9c94098861f124f (127.2.237.3:43845)
I20260620 22:59:19.273268 7117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.3@KRBTEST.COM'} at 127.2.237.3:60387
I20260620 22:59:19.276508 2996 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 22:59:19.293200 7307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:19.281062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36676 (local address 127.2.237.62:45559)
0620 22:59:19.281261 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:19.281269 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:19.281811 (+ 542us) server_negotiation.cc:407] Connection header received
0620 22:59:19.281939 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:19.281943 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:19.282006 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:19.282087 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:19.283597 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.284286 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.285477 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.285895 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.285987 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:19.288371 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:19.288375 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:19.288378 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:19.288404 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:19.290809 (+ 2405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:19.291443 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:19.291447 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:19.291449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:19.291596 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:19.291921 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:19.291924 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:19.291926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:19.292204 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:19.292393 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:19.292689 (+ 296us) server_negotiation.cc:299] Negotiation successful
0620 22:59:19.292853 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260620 22:59:19.297550 7117 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36676:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 22:59:19.304189 7117 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 22:59:19.326836 7583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:19.315900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:41575 (local address 127.0.0.1:54628)
0620 22:59:19.318286 (+ 2386us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:19.318304 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:19.318428 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:19.319421 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:19.319429 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:19.319462 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:59:19.319930 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:19.319941 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.321810 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.321822 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:19.323965 (+ 2143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:19.323975 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.326097 (+ 2122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.326101 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:19.326257 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:19.326290 (+ 33us) client_negotiation.cc:769] Sending connection context
0620 22:59:19.326373 (+ 83us) client_negotiation.cc:240] Negotiation successful
0620 22:59:19.326485 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2267,"thread_start_us":2146,"threads_started":1}
I20260620 22:59:19.326810 7585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:19.316286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52368 (local address 127.2.237.3:43845)
0620 22:59:19.318451 (+ 2165us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:19.318461 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:19.318494 (+ 33us) server_negotiation.cc:407] Connection header received
0620 22:59:19.318645 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:19.318683 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:19.318926 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:19.319771 (+ 845us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:59:19.320052 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.321434 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.324258 (+ 2824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.325600 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.325732 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:19.325961 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:19.326252 (+ 291us) server_negotiation.cc:299] Negotiation successful
0620 22:59:19.326434 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2004,"thread_start_us":1841,"threads_started":1}
I20260620 22:59:19.327649 7584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:19.316332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54628 (local address 127.2.237.1:41575)
0620 22:59:19.317959 (+ 1627us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:19.317971 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:19.318412 (+ 441us) server_negotiation.cc:407] Connection header received
0620 22:59:19.318647 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:19.318656 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:19.318937 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:19.319130 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:59:19.320131 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.321662 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.324112 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.325600 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.325732 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:19.325960 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:19.327139 (+ 1179us) server_negotiation.cc:299] Negotiation successful
0620 22:59:19.327321 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1447,"thread_start_us":1307,"threads_started":1}
I20260620 22:59:19.327745 7582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:19.315778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:43845 (local address 127.0.0.1:52368)
0620 22:59:19.317163 (+ 1385us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:19.317210 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:19.317357 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:19.319163 (+ 1806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:19.319170 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:19.319197 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:59:19.319604 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:19.319619 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.321578 (+ 1959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.321586 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:19.324104 (+ 2518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:19.324117 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.325801 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.325808 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:19.326004 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:19.326057 (+ 53us) client_negotiation.cc:769] Sending connection context
0620 22:59:19.326204 (+ 147us) client_negotiation.cc:240] Negotiation successful
0620 22:59:19.327342 (+ 1138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1154,"mutex_wait_us":29,"thread_start_us":1032,"threads_started":1}
I20260620 22:59:19.328151 7587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:19.318987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49280 (local address 127.2.237.2:37115)
0620 22:59:19.320075 (+ 1088us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:19.320084 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:19.320603 (+ 519us) server_negotiation.cc:407] Connection header received
0620 22:59:19.320779 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:19.320786 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:19.320991 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:19.321187 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:59:19.321760 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.323360 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.325240 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.326485 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.326641 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:19.326829 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:19.327654 (+ 825us) server_negotiation.cc:299] Negotiation successful
0620 22:59:19.327844 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":943,"thread_start_us":852,"threads_started":1}
I20260620 22:59:19.328528 7586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:19.318450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:37115 (local address 127.0.0.1:49280)
0620 22:59:19.320477 (+ 2027us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:19.320496 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:19.320613 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:19.321192 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:19.321197 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:19.321220 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:59:19.321583 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:19.321594 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.323508 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.323514 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:19.325087 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:19.325098 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.327325 (+ 2227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.327332 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:19.327466 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:19.327488 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 22:59:19.327599 (+ 111us) client_negotiation.cc:240] Negotiation successful
0620 22:59:19.328005 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1905,"thread_start_us":1763,"threads_started":1}
I20260620 22:59:19.331591 7505 tablet_service.cc:1467] Processing CreateTablet for tablet 7d15fc7f797e499d962d466cc01a2db6 (DEFAULT_TABLE table=test-table [id=18694f2b16da49c0877d455bf32b9bae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:59:19.331591 7240 tablet_service.cc:1467] Processing CreateTablet for tablet 7d15fc7f797e499d962d466cc01a2db6 (DEFAULT_TABLE table=test-table [id=18694f2b16da49c0877d455bf32b9bae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:59:19.331593 7373 tablet_service.cc:1467] Processing CreateTablet for tablet 7d15fc7f797e499d962d466cc01a2db6 (DEFAULT_TABLE table=test-table [id=18694f2b16da49c0877d455bf32b9bae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:59:19.332909 7373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d15fc7f797e499d962d466cc01a2db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:19.332957 7240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d15fc7f797e499d962d466cc01a2db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:19.332959 7505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d15fc7f797e499d962d466cc01a2db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:19.345681 7588 tablet_bootstrap.cc:492] T 7d15fc7f797e499d962d466cc01a2db6 P 3e567c8f4df94104b11eb64159c666af: Bootstrap starting.
I20260620 22:59:19.347038 7589 tablet_bootstrap.cc:492] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e: Bootstrap starting.
I20260620 22:59:19.348732 7590 tablet_bootstrap.cc:492] T 7d15fc7f797e499d962d466cc01a2db6 P 8f720e2177f941b8a9c94098861f124f: Bootstrap starting.
I20260620 22:59:19.349756 7588 tablet_bootstrap.cc:654] T 7d15fc7f797e499d962d466cc01a2db6 P 3e567c8f4df94104b11eb64159c666af: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:19.350423 7589 tablet_bootstrap.cc:654] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:19.351110 7588 log.cc:824] T 7d15fc7f797e499d962d466cc01a2db6 P 3e567c8f4df94104b11eb64159c666af: Log is configured to *not* fsync() on all Append() calls
I20260620 22:59:19.351718 7590 tablet_bootstrap.cc:654] T 7d15fc7f797e499d962d466cc01a2db6 P 8f720e2177f941b8a9c94098861f124f: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:19.351753 7589 log.cc:824] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e: Log is configured to *not* fsync() on all Append() calls
I20260620 22:59:19.352880 7590 log.cc:824] T 7d15fc7f797e499d962d466cc01a2db6 P 8f720e2177f941b8a9c94098861f124f: Log is configured to *not* fsync() on all Append() calls
I20260620 22:59:19.354432 7588 tablet_bootstrap.cc:492] T 7d15fc7f797e499d962d466cc01a2db6 P 3e567c8f4df94104b11eb64159c666af: No bootstrap required, opened a new log
I20260620 22:59:19.354735 7588 ts_tablet_manager.cc:1397] T 7d15fc7f797e499d962d466cc01a2db6 P 3e567c8f4df94104b11eb64159c666af: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260620 22:59:19.354905 7589 tablet_bootstrap.cc:492] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e: No bootstrap required, opened a new log
I20260620 22:59:19.355175 7589 ts_tablet_manager.cc:1397] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260620 22:59:19.356590 7590 tablet_bootstrap.cc:492] T 7d15fc7f797e499d962d466cc01a2db6 P 8f720e2177f941b8a9c94098861f124f: No bootstrap required, opened a new log
I20260620 22:59:19.356845 7590 ts_tablet_manager.cc:1397] T 7d15fc7f797e499d962d466cc01a2db6 P 8f720e2177f941b8a9c94098861f124f: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260620 22:59:19.363536 7590 raft_consensus.cc:348] T 7d15fc7f797e499d962d466cc01a2db6 P 8f720e2177f941b8a9c94098861f124f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f720e2177f941b8a9c94098861f124f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43845 } } peers { permanent_uuid: "85315db07a5e48d3be97b93b338c1d2e" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41575 } } peers { permanent_uuid: "3e567c8f4df94104b11eb64159c666af" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 37115 } }
I20260620 22:59:19.363982 7590 raft_consensus.cc:374] T 7d15fc7f797e499d962d466cc01a2db6 P 8f720e2177f941b8a9c94098861f124f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:19.364086 7590 raft_consensus.cc:729] T 7d15fc7f797e499d962d466cc01a2db6 P 8f720e2177f941b8a9c94098861f124f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f720e2177f941b8a9c94098861f124f, State: Initialized, Role: FOLLOWER
I20260620 22:59:19.364827 7590 consensus_queue.cc:260] T 7d15fc7f797e499d962d466cc01a2db6 P 8f720e2177f941b8a9c94098861f124f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f720e2177f941b8a9c94098861f124f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43845 } } peers { permanent_uuid: "85315db07a5e48d3be97b93b338c1d2e" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41575 } } peers { permanent_uuid: "3e567c8f4df94104b11eb64159c666af" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 37115 } }
I20260620 22:59:19.365605 7588 raft_consensus.cc:348] T 7d15fc7f797e499d962d466cc01a2db6 P 3e567c8f4df94104b11eb64159c666af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f720e2177f941b8a9c94098861f124f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43845 } } peers { permanent_uuid: "85315db07a5e48d3be97b93b338c1d2e" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41575 } } peers { permanent_uuid: "3e567c8f4df94104b11eb64159c666af" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 37115 } }
I20260620 22:59:19.365605 7589 raft_consensus.cc:348] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f720e2177f941b8a9c94098861f124f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43845 } } peers { permanent_uuid: "85315db07a5e48d3be97b93b338c1d2e" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41575 } } peers { permanent_uuid: "3e567c8f4df94104b11eb64159c666af" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 37115 } }
I20260620 22:59:19.366057 7588 raft_consensus.cc:374] T 7d15fc7f797e499d962d466cc01a2db6 P 3e567c8f4df94104b11eb64159c666af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:19.366079 7589 raft_consensus.cc:374] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:19.366168 7589 raft_consensus.cc:729] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85315db07a5e48d3be97b93b338c1d2e, State: Initialized, Role: FOLLOWER
I20260620 22:59:19.366173 7588 raft_consensus.cc:729] T 7d15fc7f797e499d962d466cc01a2db6 P 3e567c8f4df94104b11eb64159c666af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e567c8f4df94104b11eb64159c666af, State: Initialized, Role: FOLLOWER
I20260620 22:59:19.366297 7570 heartbeater.cc:503] Master 127.2.237.62:45559 was elected leader, sending a full tablet report...
I20260620 22:59:19.366915 7588 consensus_queue.cc:260] T 7d15fc7f797e499d962d466cc01a2db6 P 3e567c8f4df94104b11eb64159c666af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f720e2177f941b8a9c94098861f124f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43845 } } peers { permanent_uuid: "85315db07a5e48d3be97b93b338c1d2e" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41575 } } peers { permanent_uuid: "3e567c8f4df94104b11eb64159c666af" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 37115 } }
I20260620 22:59:19.366915 7589 consensus_queue.cc:260] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f720e2177f941b8a9c94098861f124f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43845 } } peers { permanent_uuid: "85315db07a5e48d3be97b93b338c1d2e" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41575 } } peers { permanent_uuid: "3e567c8f4df94104b11eb64159c666af" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 37115 } }
I20260620 22:59:19.367724 7590 ts_tablet_manager.cc:1428] T 7d15fc7f797e499d962d466cc01a2db6 P 8f720e2177f941b8a9c94098861f124f: Time spent starting tablet: real 0.011s user 0.005s sys 0.006s
I20260620 22:59:19.369143 7438 heartbeater.cc:503] Master 127.2.237.62:45559 was elected leader, sending a full tablet report...
I20260620 22:59:19.370180 7588 ts_tablet_manager.cc:1428] T 7d15fc7f797e499d962d466cc01a2db6 P 3e567c8f4df94104b11eb64159c666af: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260620 22:59:19.370976 7589 ts_tablet_manager.cc:1428] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
W20260620 22:59:19.471421 7306 tablet.cc:2369] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 22:59:19.481158 7439 tablet.cc:2369] T 7d15fc7f797e499d962d466cc01a2db6 P 3e567c8f4df94104b11eb64159c666af: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 22:59:19.497887 7571 tablet.cc:2369] T 7d15fc7f797e499d962d466cc01a2db6 P 8f720e2177f941b8a9c94098861f124f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 22:59:19.656633 7596 raft_consensus.cc:482] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 22:59:19.657151 7596 raft_consensus.cc:504] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f720e2177f941b8a9c94098861f124f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43845 } } peers { permanent_uuid: "85315db07a5e48d3be97b93b338c1d2e" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41575 } } peers { permanent_uuid: "3e567c8f4df94104b11eb64159c666af" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 37115 } }
I20260620 22:59:19.659185 7596 leader_election.cc:288] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f720e2177f941b8a9c94098861f124f (127.2.237.3:43845), 3e567c8f4df94104b11eb64159c666af (127.2.237.2:37115)
I20260620 22:59:19.665962 7587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:19.659798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:35349 (local address 127.2.237.2:37115)
0620 22:59:19.660025 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:19.660032 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:19.660593 (+ 561us) server_negotiation.cc:407] Connection header received
0620 22:59:19.660808 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:19.660812 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:19.660871 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:19.660972 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:59:19.661624 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.662329 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.664203 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.664968 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.665127 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:19.665232 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:19.665609 (+ 377us) server_negotiation.cc:299] Negotiation successful
0620 22:59:19.665710 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260620 22:59:19.666314 7585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:19.659532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:60257 (local address 127.2.237.3:43845)
0620 22:59:19.659808 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:19.659816 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:19.660567 (+ 751us) server_negotiation.cc:407] Connection header received
0620 22:59:19.660859 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:19.660863 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:19.660926 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:19.661033 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:59:19.661731 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.662457 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.664451 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.665309 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.665470 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:19.665612 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:19.665897 (+ 285us) server_negotiation.cc:299] Negotiation successful
0620 22:59:19.666023 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260620 22:59:19.666661 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:19.659389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:43845 (local address 127.2.237.1:60257)
0620 22:59:19.660421 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:19.660442 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:19.660691 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:19.661047 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:19.661051 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:19.661079 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:59:19.661577 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:19.661587 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.662641 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.662657 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:19.664281 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:19.664304 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.665552 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.665567 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:19.665737 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:19.665760 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 22:59:19.665874 (+ 114us) client_negotiation.cc:240] Negotiation successful
0620 22:59:19.666004 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":871,"mutex_wait_us":21,"spinlock_wait_cycles":22784,"thread_start_us":609,"threads_started":1}
I20260620 22:59:19.666965 7598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:19.659610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:37115 (local address 127.2.237.1:35349)
0620 22:59:19.660479 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:19.660493 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:19.660654 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:19.660987 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:19.660992 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:19.661013 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:59:19.661451 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:19.661462 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.662464 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.662472 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:19.664053 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:19.664064 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.665216 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.665222 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:19.665383 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:19.665406 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 22:59:19.666249 (+ 843us) client_negotiation.cc:240] Negotiation successful
0620 22:59:19.666403 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":775,"mutex_wait_us":85,"thread_start_us":590,"threads_started":1}
I20260620 22:59:19.667677 7525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d15fc7f797e499d962d466cc01a2db6" candidate_uuid: "85315db07a5e48d3be97b93b338c1d2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f720e2177f941b8a9c94098861f124f" is_pre_election: true
I20260620 22:59:19.668118 7525 raft_consensus.cc:2454] T 7d15fc7f797e499d962d466cc01a2db6 P 8f720e2177f941b8a9c94098861f124f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85315db07a5e48d3be97b93b338c1d2e in term 0.
I20260620 22:59:19.668001 7393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d15fc7f797e499d962d466cc01a2db6" candidate_uuid: "85315db07a5e48d3be97b93b338c1d2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e567c8f4df94104b11eb64159c666af" is_pre_election: true
I20260620 22:59:19.668578 7393 raft_consensus.cc:2454] T 7d15fc7f797e499d962d466cc01a2db6 P 3e567c8f4df94104b11eb64159c666af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85315db07a5e48d3be97b93b338c1d2e in term 0.
I20260620 22:59:19.668795 7194 leader_election.cc:302] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85315db07a5e48d3be97b93b338c1d2e, 8f720e2177f941b8a9c94098861f124f; no voters:
I20260620 22:59:19.669263 7596 raft_consensus.cc:2781] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 22:59:19.669425 7596 raft_consensus.cc:482] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 22:59:19.669539 7596 raft_consensus.cc:3037] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:19.672055 7596 raft_consensus.cc:504] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f720e2177f941b8a9c94098861f124f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43845 } } peers { permanent_uuid: "85315db07a5e48d3be97b93b338c1d2e" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41575 } } peers { permanent_uuid: "3e567c8f4df94104b11eb64159c666af" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 37115 } }
I20260620 22:59:19.672711 7596 leader_election.cc:288] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e [CANDIDATE]: Term 1 election: Requested vote from peers 8f720e2177f941b8a9c94098861f124f (127.2.237.3:43845), 3e567c8f4df94104b11eb64159c666af (127.2.237.2:37115)
I20260620 22:59:19.673094 7525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d15fc7f797e499d962d466cc01a2db6" candidate_uuid: "85315db07a5e48d3be97b93b338c1d2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f720e2177f941b8a9c94098861f124f"
I20260620 22:59:19.673223 7393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d15fc7f797e499d962d466cc01a2db6" candidate_uuid: "85315db07a5e48d3be97b93b338c1d2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e567c8f4df94104b11eb64159c666af"
I20260620 22:59:19.673321 7525 raft_consensus.cc:3037] T 7d15fc7f797e499d962d466cc01a2db6 P 8f720e2177f941b8a9c94098861f124f [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:19.673436 7393 raft_consensus.cc:3037] T 7d15fc7f797e499d962d466cc01a2db6 P 3e567c8f4df94104b11eb64159c666af [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:19.676738 7393 raft_consensus.cc:2454] T 7d15fc7f797e499d962d466cc01a2db6 P 3e567c8f4df94104b11eb64159c666af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85315db07a5e48d3be97b93b338c1d2e in term 1.
I20260620 22:59:19.676738 7525 raft_consensus.cc:2454] T 7d15fc7f797e499d962d466cc01a2db6 P 8f720e2177f941b8a9c94098861f124f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85315db07a5e48d3be97b93b338c1d2e in term 1.
I20260620 22:59:19.677151 7193 leader_election.cc:302] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e567c8f4df94104b11eb64159c666af, 85315db07a5e48d3be97b93b338c1d2e; no voters:
I20260620 22:59:19.677438 7596 raft_consensus.cc:2781] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:59:19.677955 7596 raft_consensus.cc:686] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e [term 1 LEADER]: Becoming Leader. State: Replica: 85315db07a5e48d3be97b93b338c1d2e, State: Running, Role: LEADER
I20260620 22:59:19.678591 7596 consensus_queue.cc:237] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f720e2177f941b8a9c94098861f124f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43845 } } peers { permanent_uuid: "85315db07a5e48d3be97b93b338c1d2e" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41575 } } peers { permanent_uuid: "3e567c8f4df94104b11eb64159c666af" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 37115 } }
I20260620 22:59:19.683252 7117 catalog_manager.cc:5495] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e reported cstate change: term changed from 0 to 1, leader changed from <none> to 85315db07a5e48d3be97b93b338c1d2e (127.2.237.1). New cstate: current_term: 1 leader_uuid: "85315db07a5e48d3be97b93b338c1d2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f720e2177f941b8a9c94098861f124f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85315db07a5e48d3be97b93b338c1d2e" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41575 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e567c8f4df94104b11eb64159c666af" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 37115 } health_report { overall_health: UNKNOWN } } }
I20260620 22:59:19.779953 7584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:19.774716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54640 (local address 127.2.237.1:41575)
0620 22:59:19.774928 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:19.774935 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:19.775076 (+ 141us) server_negotiation.cc:407] Connection header received
0620 22:59:19.775234 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:19.775237 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:19.775317 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:19.775456 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 22:59:19.775743 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.776439 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.777752 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.778207 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.778303 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:19.778826 (+ 523us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 22:59:19.779428 (+ 602us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 22:59:19.779534 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:19.779609 (+ 75us) server_negotiation.cc:299] Negotiation successful
0620 22:59:19.779669 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 22:59:19.787771 7525 raft_consensus.cc:1261] T 7d15fc7f797e499d962d466cc01a2db6 P 8f720e2177f941b8a9c94098861f124f [term 1 FOLLOWER]: Refusing update from remote peer 85315db07a5e48d3be97b93b338c1d2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:59:19.787801 7393 raft_consensus.cc:1261] T 7d15fc7f797e499d962d466cc01a2db6 P 3e567c8f4df94104b11eb64159c666af [term 1 FOLLOWER]: Refusing update from remote peer 85315db07a5e48d3be97b93b338c1d2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:59:19.788722 7599 consensus_queue.cc:1035] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f720e2177f941b8a9c94098861f124f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:59:19.789034 7596 consensus_queue.cc:1035] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e567c8f4df94104b11eb64159c666af" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 37115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:59:19.799885 7602 mvcc.cc:205] Tried to move back new op lower bound from 7299057089681190912 to 7299057089248235520. Current Snapshot: MvccSnapshot[applied={T|T < 7299057089681190912}]
I20260620 22:59:19.800467 7604 mvcc.cc:205] Tried to move back new op lower bound from 7299057089681190912 to 7299057089248235520. Current Snapshot: MvccSnapshot[applied={T|T < 7299057089681190912}]
I20260620 22:59:19.813805 7116 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 22:59:19.814013 7116 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 22:59:19.817409 7116 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5b46bf13296d4fa4aec640a8ff6843a6: Sending DeleteTablet for 3 replicas of tablet 7d15fc7f797e499d962d466cc01a2db6
I20260620 22:59:19.818648 7240 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d15fc7f797e499d962d466cc01a2db6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:19 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:54628
I20260620 22:59:19.818646 7505 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d15fc7f797e499d962d466cc01a2db6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:19 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:52368
I20260620 22:59:19.818650 7373 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d15fc7f797e499d962d466cc01a2db6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:19 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:49280
I20260620 22:59:19.820125 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 7178
I20260620 22:59:19.831368 7614 tablet_replica.cc:330] T 7d15fc7f797e499d962d466cc01a2db6 P 3e567c8f4df94104b11eb64159c666af: stopping tablet replica
I20260620 22:59:19.832749 7614 raft_consensus.cc:2229] T 7d15fc7f797e499d962d466cc01a2db6 P 3e567c8f4df94104b11eb64159c666af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 22:59:19.834115 7614 raft_consensus.cc:2258] T 7d15fc7f797e499d962d466cc01a2db6 P 3e567c8f4df94104b11eb64159c666af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:19.838027 7614 ts_tablet_manager.cc:1905] T 7d15fc7f797e499d962d466cc01a2db6 P 3e567c8f4df94104b11eb64159c666af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:19.839417 7613 tablet_replica.cc:330] T 7d15fc7f797e499d962d466cc01a2db6 P 8f720e2177f941b8a9c94098861f124f: stopping tablet replica
I20260620 22:59:19.840050 7612 tablet_replica.cc:330] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e: stopping tablet replica
I20260620 22:59:19.840535 7612 raft_consensus.cc:2229] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e [term 1 LEADER]: Raft consensus shutting down.
I20260620 22:59:19.841002 7613 raft_consensus.cc:2229] T 7d15fc7f797e499d962d466cc01a2db6 P 8f720e2177f941b8a9c94098861f124f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 22:59:19.841789 7612 raft_consensus.cc:2258] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:19.842460 7613 raft_consensus.cc:2258] T 7d15fc7f797e499d962d466cc01a2db6 P 8f720e2177f941b8a9c94098861f124f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:19.844921 7612 ts_tablet_manager.cc:1905] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e: Deleting tablet data with delete state TABLET_DATA_DELETED
Jun 20 22:59:19 dist-test-slave-f63b krb5kdc[7073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.1@KRBTEST.COM
I20260620 22:59:19.846655 7613 ts_tablet_manager.cc:1905] T 7d15fc7f797e499d962d466cc01a2db6 P 8f720e2177f941b8a9c94098861f124f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:19.846920 7614 ts_tablet_manager.cc:1918] T 7d15fc7f797e499d962d466cc01a2db6 P 3e567c8f4df94104b11eb64159c666af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 22:59:19.847075 7614 log.cc:1196] T 7d15fc7f797e499d962d466cc01a2db6 P 3e567c8f4df94104b11eb64159c666af: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-1/wal/wals/7d15fc7f797e499d962d466cc01a2db6
I20260620 22:59:19.847605 7614 ts_tablet_manager.cc:1939] T 7d15fc7f797e499d962d466cc01a2db6 P 3e567c8f4df94104b11eb64159c666af: Deleting consensus metadata
I20260620 22:59:19.849320 7101 catalog_manager.cc:4841] TS 3e567c8f4df94104b11eb64159c666af (127.2.237.2:37115): tablet 7d15fc7f797e499d962d466cc01a2db6 (table test-table [id=18694f2b16da49c0877d455bf32b9bae]) successfully deleted
I20260620 22:59:19.855393 7584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:19.821861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54644 (local address 127.2.237.1:41575)
0620 22:59:19.822038 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:19.822044 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:19.835636 (+ 13592us) server_negotiation.cc:407] Connection header received
0620 22:59:19.836152 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:19.836159 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:19.836251 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:19.836327 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:19.838041 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.839393 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.841312 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.842034 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.842137 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:19.847557 (+ 5420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:19.847573 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:19.847595 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:19.847651 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:19.851839 (+ 4188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:19.852659 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:19.852674 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:19.852678 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:19.852887 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:19.853358 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:19.853363 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:19.853367 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:19.853825 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:19.854124 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:19.854746 (+ 622us) server_negotiation.cc:299] Negotiation successful
0620 22:59:19.854986 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260620 22:59:19.856149 7612 ts_tablet_manager.cc:1918] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 22:59:19.856366 7612 log.cc:1196] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-0/wal/wals/7d15fc7f797e499d962d466cc01a2db6
I20260620 22:59:19.856346 7613 ts_tablet_manager.cc:1918] T 7d15fc7f797e499d962d466cc01a2db6 P 8f720e2177f941b8a9c94098861f124f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 22:59:19.856490 7613 log.cc:1196] T 7d15fc7f797e499d962d466cc01a2db6 P 8f720e2177f941b8a9c94098861f124f: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781996318807807-2996-0/minicluster-data/ts-2/wal/wals/7d15fc7f797e499d962d466cc01a2db6
I20260620 22:59:19.856819 7300 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:19.857010 7613 ts_tablet_manager.cc:1939] T 7d15fc7f797e499d962d466cc01a2db6 P 8f720e2177f941b8a9c94098861f124f: Deleting consensus metadata
I20260620 22:59:19.858958 7102 catalog_manager.cc:4841] TS 8f720e2177f941b8a9c94098861f124f (127.2.237.3:43845): tablet 7d15fc7f797e499d962d466cc01a2db6 (table test-table [id=18694f2b16da49c0877d455bf32b9bae]) successfully deleted
I20260620 22:59:19.953025 7612 ts_tablet_manager.cc:1939] T 7d15fc7f797e499d962d466cc01a2db6 P 85315db07a5e48d3be97b93b338c1d2e: Deleting consensus metadata
I20260620 22:59:19.953692 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 7178
I20260620 22:59:19.968668 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 7311
W20260620 22:59:19.968595 7101 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.237.1:41575 (error 108)
I20260620 22:59:19.970844 7618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:19.970063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:41575 (local address 127.0.0.1:54660)
0620 22:59:19.970501 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:19.970673 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:41575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":222,"threads_started":1}
W20260620 22:59:19.971110 7101 catalog_manager.cc:4584] TS 85315db07a5e48d3be97b93b338c1d2e (127.2.237.1:41575): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7d15fc7f797e499d962d466cc01a2db6: Network error: Client connection negotiation failed: client connection to 127.2.237.1:41575: connect: Connection refused (error 111)
Jun 20 22:59:19 dist-test-slave-f63b krb5kdc[7073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.2@KRBTEST.COM
I20260620 22:59:19.983983 7587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:19.969506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49288 (local address 127.2.237.2:37115)
0620 22:59:19.969672 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:19.969676 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:19.969754 (+ 78us) server_negotiation.cc:407] Connection header received
0620 22:59:19.969882 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:19.969886 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:19.969956 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:19.970042 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:19.971661 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.972423 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.973601 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:19.973956 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:19.974044 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:19.978423 (+ 4379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:19.978436 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:19.978452 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:19.978487 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:19.981257 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:19.981899 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:19.981906 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:19.981908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:19.982041 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:19.982439 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:19.982443 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:19.982446 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:19.982946 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:19.983157 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:19.983476 (+ 319us) server_negotiation.cc:299] Negotiation successful
0620 22:59:19.983634 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 22:59:19.984654 7433 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:20.024525 7618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:20.024083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:41575 (local address 127.0.0.1:54670)
0620 22:59:20.024300 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:20.024382 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:41575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260620 22:59:20.025558 7618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:20.025196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:41575 (local address 127.0.0.1:54686)
0620 22:59:20.025371 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:20.025434 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:41575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260620 22:59:20.073422 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 7311
I20260620 22:59:20.086424 7618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:20.085829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:41575 (local address 127.0.0.1:54696)
0620 22:59:20.086083 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:20.086204 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:41575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260620 22:59:20.087725 7618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:20.087256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:41575 (local address 127.0.0.1:54710)
0620 22:59:20.087444 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:20.087537 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:41575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260620 22:59:20.090396 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 7443
Jun 20 22:59:20 dist-test-slave-f63b krb5kdc[7073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.3@KRBTEST.COM
I20260620 22:59:20.104911 7585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:20.091183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52372 (local address 127.2.237.3:43845)
0620 22:59:20.091376 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:20.091382 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:20.091402 (+ 20us) server_negotiation.cc:407] Connection header received
0620 22:59:20.091571 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:20.091577 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:20.091653 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:20.091755 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:20.092976 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:20.093673 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:20.095056 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:20.095460 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:20.095591 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:20.099627 (+ 4036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:20.099641 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:20.099657 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:20.099693 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:20.102179 (+ 2486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:20.102926 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:20.102935 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:20.102938 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:20.103077 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:20.103450 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:20.103455 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:20.103460 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:20.103797 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:20.104085 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:20.104376 (+ 291us) server_negotiation.cc:299] Negotiation successful
0620 22:59:20.104589 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260620 22:59:20.105511 7565 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:20.168507 7618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:20.167963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:41575 (local address 127.0.0.1:54716)
0620 22:59:20.168207 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:20.168343 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:41575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260620 22:59:20.169670 7618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:20.169227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:41575 (local address 127.0.0.1:54718)
0620 22:59:20.169425 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:20.169509 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:41575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260620 22:59:20.195217 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 7443
I20260620 22:59:20.211300 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 7086
I20260620 22:59:20.212378 7147 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:20.321190 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 7086
2026-06-20T22:59:20Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4420 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-20T22:59:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T22:59:20Z Disabled control of system clock
I20260620 22:59:20.387491 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:33373
--webserver_interface=127.2.237.62
--webserver_port=0
--builtin_ntp_servers=127.2.237.20:37651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:33373
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 22:59:20.702098 7625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:20.702428 7625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:20.702595 7625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:20.710843 7625 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 22:59:20.710979 7625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:20.711045 7625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:20.711123 7625 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 22:59:20.711187 7625 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 22:59:20.720413 7625 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:37651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:33373
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:33373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:20.721673 7625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:20.723524 7625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:20.735250 7630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:20.735213 7633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:20.735272 7631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:20.736203 7625 server_base.cc:1034] running on GCE node
I20260620 22:59:20.737010 7625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:20.738346 7625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:20.739568 7625 hybrid_clock.cc:648] HybridClock initialized: now 1781996360739503 us; error 84 us; skew 500 ppm
I20260620 22:59:20.739940 7625 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 22:59:20.742161 7625 webserver.cc:460] Webserver started at http://127.2.237.62:45467/ using document root <none> and password file <none>
I20260620 22:59:20.742929 7625 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:20.743057 7625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:20.743376 7625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:20.746232 7625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "9ba8e08954a2456ca9032a7f9a137e3b"
format_stamp: "Formatted at 2026-06-20 22:59:20 on dist-test-slave-f63b"
server_key: "1d632e042085a56239e6ded8c97d888f"
server_key_iv: "e6374861dc0d01656534ee8acf95c42f"
server_key_version: "encryptionkey@0"
I20260620 22:59:20.747056 7625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "9ba8e08954a2456ca9032a7f9a137e3b"
format_stamp: "Formatted at 2026-06-20 22:59:20 on dist-test-slave-f63b"
server_key: "1d632e042085a56239e6ded8c97d888f"
server_key_iv: "e6374861dc0d01656534ee8acf95c42f"
server_key_version: "encryptionkey@0"
I20260620 22:59:20.751892 7625 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260620 22:59:20.755155 7639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:20.756366 7625 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 22:59:20.756557 7625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "9ba8e08954a2456ca9032a7f9a137e3b"
format_stamp: "Formatted at 2026-06-20 22:59:20 on dist-test-slave-f63b"
server_key: "1d632e042085a56239e6ded8c97d888f"
server_key_iv: "e6374861dc0d01656534ee8acf95c42f"
server_key_version: "encryptionkey@0"
I20260620 22:59:20.756778 7625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:20.762157 7625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:20.762934 7625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:20.763211 7625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:20.780875 7625 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:33373
I20260620 22:59:20.780910 7690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:33373 every 8 connection(s)
I20260620 22:59:20.782158 7625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 22:59:20.784729 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 7625
I20260620 22:59:20.784935 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 22:59:20.785563 2996 external_mini_cluster.cc:1428] Setting key 3749042e0aaf8f4813ccf4f2e357a2a5
I20260620 22:59:20.785925 7691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:20.793766 7694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:20.787510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56390 (local address 127.2.237.62:33373)
0620 22:59:20.788508 (+ 998us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:20.788519 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:20.788562 (+ 43us) server_negotiation.cc:407] Connection header received
0620 22:59:20.790326 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:20.790346 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:20.790830 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:20.791152 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:20.792241 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:20.792246 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:20.792258 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 22:59:20.792272 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:20.792337 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 22:59:20.792352 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:20.792507 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:20.792618 (+ 111us) server_negotiation.cc:299] Negotiation successful
0620 22:59:20.792897 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":355,"threads_started":1}
I20260620 22:59:20.798851 7691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ba8e08954a2456ca9032a7f9a137e3b: Bootstrap starting.
I20260620 22:59:20.801803 7691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ba8e08954a2456ca9032a7f9a137e3b: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:20.802811 7691 log.cc:824] T 00000000000000000000000000000000 P 9ba8e08954a2456ca9032a7f9a137e3b: Log is configured to *not* fsync() on all Append() calls
I20260620 22:59:20.805722 7691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ba8e08954a2456ca9032a7f9a137e3b: No bootstrap required, opened a new log
I20260620 22:59:20.813162 7691 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ba8e08954a2456ca9032a7f9a137e3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba8e08954a2456ca9032a7f9a137e3b" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 33373 } }
I20260620 22:59:20.813537 7691 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ba8e08954a2456ca9032a7f9a137e3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:20.813655 7691 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ba8e08954a2456ca9032a7f9a137e3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ba8e08954a2456ca9032a7f9a137e3b, State: Initialized, Role: FOLLOWER
I20260620 22:59:20.814215 7691 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ba8e08954a2456ca9032a7f9a137e3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba8e08954a2456ca9032a7f9a137e3b" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 33373 } }
I20260620 22:59:20.814465 7691 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ba8e08954a2456ca9032a7f9a137e3b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 22:59:20.814641 7691 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ba8e08954a2456ca9032a7f9a137e3b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 22:59:20.814838 7691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ba8e08954a2456ca9032a7f9a137e3b [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:20.817006 7691 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ba8e08954a2456ca9032a7f9a137e3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba8e08954a2456ca9032a7f9a137e3b" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 33373 } }
I20260620 22:59:20.817520 7691 leader_election.cc:302] T 00000000000000000000000000000000 P 9ba8e08954a2456ca9032a7f9a137e3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ba8e08954a2456ca9032a7f9a137e3b; no voters:
I20260620 22:59:20.818174 7691 leader_election.cc:288] T 00000000000000000000000000000000 P 9ba8e08954a2456ca9032a7f9a137e3b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 22:59:20.818331 7696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ba8e08954a2456ca9032a7f9a137e3b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:59:20.819489 7696 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ba8e08954a2456ca9032a7f9a137e3b [term 1 LEADER]: Becoming Leader. State: Replica: 9ba8e08954a2456ca9032a7f9a137e3b, State: Running, Role: LEADER
I20260620 22:59:20.820029 7696 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ba8e08954a2456ca9032a7f9a137e3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba8e08954a2456ca9032a7f9a137e3b" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 33373 } }
I20260620 22:59:20.820632 7691 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ba8e08954a2456ca9032a7f9a137e3b [sys.catalog]: configured and running, proceeding with master startup.
I20260620 22:59:20.822873 7698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ba8e08954a2456ca9032a7f9a137e3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ba8e08954a2456ca9032a7f9a137e3b. Latest consensus state: current_term: 1 leader_uuid: "9ba8e08954a2456ca9032a7f9a137e3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba8e08954a2456ca9032a7f9a137e3b" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 33373 } } }
I20260620 22:59:20.823220 7698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ba8e08954a2456ca9032a7f9a137e3b [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:20.822956 7697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ba8e08954a2456ca9032a7f9a137e3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ba8e08954a2456ca9032a7f9a137e3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ba8e08954a2456ca9032a7f9a137e3b" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 33373 } } }
I20260620 22:59:20.823388 7697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ba8e08954a2456ca9032a7f9a137e3b [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:20.826367 7701 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 22:59:20.830544 7701 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 22:59:20.838730 7701 catalog_manager.cc:1347] Generated new cluster ID: a1714e9e0efa4ce1b12f6e77f3ee8ad7
I20260620 22:59:20.838838 7701 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 22:59:20.861846 7701 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 22:59:20.862277 7701 catalog_manager.cc:1504] Loading token signing keys...
I20260620 22:59:20.876017 7701 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ba8e08954a2456ca9032a7f9a137e3b: Generated new TSK 0
I20260620 22:59:20.877091 7701 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 22:59:20.891350 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:0
--local_ip_for_outbound_sockets=127.2.237.1
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:33373
--builtin_ntp_servers=127.2.237.20:37651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260620 22:59:21.228422 7715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:21.228739 7715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:21.228844 7715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:21.236449 7715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:21.236580 7715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:21.236722 7715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 22:59:21.246114 7715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:37651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:33373
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:21.247437 7715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:21.249416 7715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:21.263515 7721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:21.263639 7720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:21.263931 7723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:21.264694 7715 server_base.cc:1034] running on GCE node
I20260620 22:59:21.265501 7715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:21.266467 7715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:21.267693 7715 hybrid_clock.cc:648] HybridClock initialized: now 1781996361267624 us; error 57 us; skew 500 ppm
I20260620 22:59:21.268045 7715 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 22:59:21.270907 7715 webserver.cc:460] Webserver started at http://127.2.237.1:34885/ using document root <none> and password file <none>
I20260620 22:59:21.271634 7715 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:21.271747 7715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:21.272060 7715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:21.275103 7715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-0/data/instance:
uuid: "fae520cec6d34a4b8294d40c1871a9b9"
format_stamp: "Formatted at 2026-06-20 22:59:21 on dist-test-slave-f63b"
server_key: "6ad4c5e8b0bdd2ac78cbc781a2d22d7c"
server_key_iv: "f150a9c0562e49fbde4bef2959719e82"
server_key_version: "encryptionkey@0"
I20260620 22:59:21.275938 7715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance:
uuid: "fae520cec6d34a4b8294d40c1871a9b9"
format_stamp: "Formatted at 2026-06-20 22:59:21 on dist-test-slave-f63b"
server_key: "6ad4c5e8b0bdd2ac78cbc781a2d22d7c"
server_key_iv: "f150a9c0562e49fbde4bef2959719e82"
server_key_version: "encryptionkey@0"
I20260620 22:59:21.282220 7715 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260620 22:59:21.285590 7729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:21.287036 7715 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260620 22:59:21.287210 7715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "fae520cec6d34a4b8294d40c1871a9b9"
format_stamp: "Formatted at 2026-06-20 22:59:21 on dist-test-slave-f63b"
server_key: "6ad4c5e8b0bdd2ac78cbc781a2d22d7c"
server_key_iv: "f150a9c0562e49fbde4bef2959719e82"
server_key_version: "encryptionkey@0"
I20260620 22:59:21.287442 7715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:21.293089 7715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:21.293866 7715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:21.294131 7715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:21.295125 7715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:59:21.296471 7715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:21.296597 7715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:21.296725 7715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:21.296811 7715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:21.329489 7715 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:33013
I20260620 22:59:21.329509 7841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:33013 every 8 connection(s)
I20260620 22:59:21.331072 7715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 22:59:21.339234 7844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:21.333736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:34077 (local address 127.2.237.62:33373)
0620 22:59:21.334138 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:21.334143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:21.335553 (+ 1410us) server_negotiation.cc:407] Connection header received
0620 22:59:21.337134 (+ 1581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:21.337139 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:21.337196 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:21.337293 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:21.338625 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:21.338630 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:21.338635 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 22:59:21.338655 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:21.338695 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 22:59:21.338709 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:21.338808 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:21.338968 (+ 160us) server_negotiation.cc:299] Negotiation successful
0620 22:59:21.339013 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":209,"threads_started":1}
I20260620 22:59:21.340126 7845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:21.334397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:33373 (local address 127.2.237.1:34077)
0620 22:59:21.335390 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:21.335431 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:21.336897 (+ 1466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:21.337506 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:21.337513 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:21.337604 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:21.338309 (+ 705us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 22:59:21.338311 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:21.338350 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 22:59:21.338432 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:21.338796 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:21.338803 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:21.338809 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 22:59:21.339020 (+ 211us) client_negotiation.cc:240] Negotiation successful
0620 22:59:21.339213 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":351,"threads_started":1}
I20260620 22:59:21.340780 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 7715
I20260620 22:59:21.340996 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance
I20260620 22:59:21.341735 7842 heartbeater.cc:347] Connected to a master server at 127.2.237.62:33373
I20260620 22:59:21.341850 2996 external_mini_cluster.cc:1428] Setting key 40feefc29a97f88652e1edab88f80756
I20260620 22:59:21.342051 7842 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:21.342643 7842 heartbeater.cc:511] Master 127.2.237.62:33373 requested a full tablet report, sending...
I20260620 22:59:21.344769 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:0
--local_ip_for_outbound_sockets=127.2.237.2
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:33373
--builtin_ntp_servers=127.2.237.20:37651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260620 22:59:21.344836 7656 ts_manager.cc:194] Registered new tserver with Master: fae520cec6d34a4b8294d40c1871a9b9 (127.2.237.1:33013)
W20260620 22:59:21.661440 7846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:21.661835 7846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:21.661944 7846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:21.669350 7846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:21.669450 7846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:21.669559 7846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 22:59:21.678750 7846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:37651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:33373
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:21.679996 7846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:21.681701 7846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:21.694919 7851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:21.694991 7852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:21.694916 7854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:21.695906 7846 server_base.cc:1034] running on GCE node
I20260620 22:59:21.696436 7846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:21.697283 7846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:21.698485 7846 hybrid_clock.cc:648] HybridClock initialized: now 1781996361698405 us; error 79 us; skew 500 ppm
I20260620 22:59:21.698858 7846 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 22:59:21.701107 7846 webserver.cc:460] Webserver started at http://127.2.237.2:35541/ using document root <none> and password file <none>
I20260620 22:59:21.701857 7846 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:21.701941 7846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:21.702284 7846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:21.705091 7846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-1/data/instance:
uuid: "bd548ce251c24115a9dd265f32d923c0"
format_stamp: "Formatted at 2026-06-20 22:59:21 on dist-test-slave-f63b"
server_key: "a0507afc004fe3cedaffa7cad75970c0"
server_key_iv: "9d5f6ce052ca043c733f3d3e86e87d5a"
server_key_version: "encryptionkey@0"
I20260620 22:59:21.705840 7846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance:
uuid: "bd548ce251c24115a9dd265f32d923c0"
format_stamp: "Formatted at 2026-06-20 22:59:21 on dist-test-slave-f63b"
server_key: "a0507afc004fe3cedaffa7cad75970c0"
server_key_iv: "9d5f6ce052ca043c733f3d3e86e87d5a"
server_key_version: "encryptionkey@0"
I20260620 22:59:21.710734 7846 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260620 22:59:21.714099 7860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:21.715415 7846 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 22:59:21.715582 7846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "bd548ce251c24115a9dd265f32d923c0"
format_stamp: "Formatted at 2026-06-20 22:59:21 on dist-test-slave-f63b"
server_key: "a0507afc004fe3cedaffa7cad75970c0"
server_key_iv: "9d5f6ce052ca043c733f3d3e86e87d5a"
server_key_version: "encryptionkey@0"
I20260620 22:59:21.715848 7846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:21.721378 7846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:21.722230 7846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:21.722488 7846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:21.723397 7846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:59:21.724727 7846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:21.724845 7846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:21.724979 7846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:21.725072 7846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:21.757124 7846 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:40815
I20260620 22:59:21.757174 7972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:40815 every 8 connection(s)
I20260620 22:59:21.758479 7846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 22:59:21.762989 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 7846
I20260620 22:59:21.763178 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance
I20260620 22:59:21.763741 2996 external_mini_cluster.cc:1428] Setting key 8a7a50d62a65c9e4f0d58de0fd735aea
I20260620 22:59:21.766392 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:0
--local_ip_for_outbound_sockets=127.2.237.3
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:33373
--builtin_ntp_servers=127.2.237.20:37651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260620 22:59:21.767611 7844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:21.761323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:59677 (local address 127.2.237.62:33373)
0620 22:59:21.761496 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:21.761501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:21.763163 (+ 1662us) server_negotiation.cc:407] Connection header received
0620 22:59:21.765314 (+ 2151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:21.765320 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:21.765402 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:21.765525 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:21.766902 (+ 1377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:21.766908 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:21.766913 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 22:59:21.766931 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:21.766970 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 22:59:21.766982 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:21.767091 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:21.767288 (+ 197us) server_negotiation.cc:299] Negotiation successful
0620 22:59:21.767350 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260620 22:59:21.768571 7975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:21.761911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:33373 (local address 127.2.237.2:59677)
0620 22:59:21.762983 (+ 1072us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:21.763031 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:21.765062 (+ 2031us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:21.765754 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:21.765763 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:21.765852 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:21.766556 (+ 704us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 22:59:21.766560 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:21.766633 (+ 73us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 22:59:21.766742 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:21.767079 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:21.767088 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:21.767097 (+ 9us) client_negotiation.cc:769] Sending connection context
0620 22:59:21.767372 (+ 275us) client_negotiation.cc:240] Negotiation successful
0620 22:59:21.767629 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":275,"threads_started":1}
I20260620 22:59:21.769976 7973 heartbeater.cc:347] Connected to a master server at 127.2.237.62:33373
I20260620 22:59:21.770260 7973 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:21.770843 7973 heartbeater.cc:511] Master 127.2.237.62:33373 requested a full tablet report, sending...
I20260620 22:59:21.772250 7656 ts_manager.cc:194] Registered new tserver with Master: bd548ce251c24115a9dd265f32d923c0 (127.2.237.2:40815)
W20260620 22:59:22.089775 7976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:22.090094 7976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:22.090276 7976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:22.097779 7976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:22.097918 7976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:22.098100 7976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 22:59:22.107506 7976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:37651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:33373
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:22.108827 7976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:22.110553 7976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:22.123706 7981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:22.123756 7982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:22.123793 7984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:22.124909 7976 server_base.cc:1034] running on GCE node
I20260620 22:59:22.125566 7976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:22.126617 7976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:22.127833 7976 hybrid_clock.cc:648] HybridClock initialized: now 1781996362127781 us; error 49 us; skew 500 ppm
I20260620 22:59:22.128180 7976 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 22:59:22.130458 7976 webserver.cc:460] Webserver started at http://127.2.237.3:37291/ using document root <none> and password file <none>
I20260620 22:59:22.131187 7976 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:22.131305 7976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:22.131631 7976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:22.134509 7976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-2/data/instance:
uuid: "38540c7404a140d9b43eb24cba8c15c4"
format_stamp: "Formatted at 2026-06-20 22:59:22 on dist-test-slave-f63b"
server_key: "4e4d9b26bcd58160a97b65a7a7840bbc"
server_key_iv: "6f96c8ed9274b3b71feb177156fa47f8"
server_key_version: "encryptionkey@0"
I20260620 22:59:22.135396 7976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance:
uuid: "38540c7404a140d9b43eb24cba8c15c4"
format_stamp: "Formatted at 2026-06-20 22:59:22 on dist-test-slave-f63b"
server_key: "4e4d9b26bcd58160a97b65a7a7840bbc"
server_key_iv: "6f96c8ed9274b3b71feb177156fa47f8"
server_key_version: "encryptionkey@0"
I20260620 22:59:22.140432 7976 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260620 22:59:22.143863 7990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:22.145153 7976 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 22:59:22.145328 7976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "38540c7404a140d9b43eb24cba8c15c4"
format_stamp: "Formatted at 2026-06-20 22:59:22 on dist-test-slave-f63b"
server_key: "4e4d9b26bcd58160a97b65a7a7840bbc"
server_key_iv: "6f96c8ed9274b3b71feb177156fa47f8"
server_key_version: "encryptionkey@0"
I20260620 22:59:22.145552 7976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:22.151268 7976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:22.152041 7976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:22.152310 7976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:22.153189 7976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:59:22.154522 7976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:22.154657 7976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:22.154798 7976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:22.154887 7976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:22.188627 7976 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:34437
I20260620 22:59:22.188717 8102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:34437 every 8 connection(s)
I20260620 22:59:22.189939 7976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 22:59:22.196488 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 7976
I20260620 22:59:22.196655 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance
I20260620 22:59:22.197252 2996 external_mini_cluster.cc:1428] Setting key 6467b10c96ffab4a83514f8d8dae2196
I20260620 22:59:22.198398 7844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:22.192569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:39391 (local address 127.2.237.62:33373)
0620 22:59:22.192765 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:22.192770 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:22.194094 (+ 1324us) server_negotiation.cc:407] Connection header received
0620 22:59:22.195754 (+ 1660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:22.195759 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:22.195816 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:22.195901 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:22.197404 (+ 1503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:22.197409 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:22.197412 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 22:59:22.197428 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:22.197458 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 22:59:22.197468 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:22.197559 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:22.198049 (+ 490us) server_negotiation.cc:299] Negotiation successful
0620 22:59:22.198109 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260620 22:59:22.199247 8105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:22.193008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:33373 (local address 127.2.237.3:39391)
0620 22:59:22.193925 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:22.193968 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:22.195511 (+ 1543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:22.196141 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:22.196154 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:22.196278 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:22.197038 (+ 760us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 22:59:22.197041 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:22.197091 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 22:59:22.197195 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:22.197572 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:22.197584 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:22.197593 (+ 9us) client_negotiation.cc:769] Sending connection context
0620 22:59:22.197870 (+ 277us) client_negotiation.cc:240] Negotiation successful
0620 22:59:22.198051 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":265,"threads_started":1}
I20260620 22:59:22.200636 8103 heartbeater.cc:347] Connected to a master server at 127.2.237.62:33373
I20260620 22:59:22.200912 8103 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:22.201449 8103 heartbeater.cc:511] Master 127.2.237.62:33373 requested a full tablet report, sending...
I20260620 22:59:22.202731 7656 ts_manager.cc:194] Registered new tserver with Master: 38540c7404a140d9b43eb24cba8c15c4 (127.2.237.3:34437)
I20260620 22:59:22.212500 2996 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 22:59:22.219285 7844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:22.216951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56406 (local address 127.2.237.62:33373)
0620 22:59:22.217085 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:22.217090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:22.217562 (+ 472us) server_negotiation.cc:407] Connection header received
0620 22:59:22.217691 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:22.217695 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:22.217747 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:22.217828 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:22.218673 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:22.218677 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:22.218680 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 22:59:22.218694 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:22.218722 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 22:59:22.218733 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:22.218824 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:22.218917 (+ 93us) server_negotiation.cc:299] Negotiation successful
0620 22:59:22.218974 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 22:59:22.223209 7656 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56406:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 22:59:22.230229 7656 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 22:59:22.262780 8117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:22.242440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38572 (local address 127.2.237.1:33013)
0620 22:59:22.258745 (+ 16305us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:22.258753 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:22.259331 (+ 578us) server_negotiation.cc:407] Connection header received
0620 22:59:22.259415 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:22.259425 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:22.259726 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:22.260509 (+ 783us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:22.261946 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:22.261954 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:22.261968 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 22:59:22.261985 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:22.262039 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 22:59:22.262060 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:22.262221 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:22.262297 (+ 76us) server_negotiation.cc:299] Negotiation successful
0620 22:59:22.262394 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16136,"thread_start_us":16308,"threads_started":1}
I20260620 22:59:22.262820 8115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:22.242166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:33013 (local address 127.0.0.1:38572)
0620 22:59:22.258807 (+ 16641us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:22.258823 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:22.258926 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:22.259942 (+ 1016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:22.259949 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:22.260051 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:22.260871 (+ 820us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 22:59:22.260874 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:22.260916 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 22:59:22.261006 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:22.262178 (+ 1172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:22.262183 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:22.262188 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 22:59:22.262320 (+ 132us) client_negotiation.cc:240] Negotiation successful
0620 22:59:22.262394 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16507,"mutex_wait_us":166,"thread_start_us":16802,"threads_started":1}
I20260620 22:59:22.263386 8118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:22.242166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:34437 (local address 127.0.0.1:42712)
0620 22:59:22.252667 (+ 10501us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:22.252705 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:22.252840 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:22.261217 (+ 8377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:22.261221 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:22.261244 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:22.261922 (+ 678us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 22:59:22.261926 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:22.261941 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 22:59:22.261977 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:22.262786 (+ 809us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:22.262791 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:22.262796 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 22:59:22.262919 (+ 123us) client_negotiation.cc:240] Negotiation successful
0620 22:59:22.262975 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10239,"thread_start_us":16369,"threads_started":1}
I20260620 22:59:22.263373 8120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:22.243279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59748 (local address 127.2.237.2:40815)
0620 22:59:22.258167 (+ 14888us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:22.258180 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:22.259514 (+ 1334us) server_negotiation.cc:407] Connection header received
0620 22:59:22.259614 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:22.259625 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:22.259896 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:22.260056 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:22.261734 (+ 1678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:22.261743 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:22.261758 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 22:59:22.261783 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:22.261841 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 22:59:22.261862 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:22.262025 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:22.262453 (+ 428us) server_negotiation.cc:299] Negotiation successful
0620 22:59:22.262620 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14642,"thread_start_us":14889,"threads_started":1}
I20260620 22:59:22.264029 8116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:22.242613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:40815 (local address 127.0.0.1:59748)
0620 22:59:22.259368 (+ 16755us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:22.259384 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:22.259478 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:22.260521 (+ 1043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:22.260526 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:22.260551 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:22.261489 (+ 938us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 22:59:22.261492 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:22.261513 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 22:59:22.261557 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:22.262085 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:22.262098 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:22.262107 (+ 9us) client_negotiation.cc:769] Sending connection context
0620 22:59:22.262297 (+ 190us) client_negotiation.cc:240] Negotiation successful
0620 22:59:22.263578 (+ 1281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16651,"mutex_wait_us":1174,"thread_start_us":9627,"threads_started":1}
I20260620 22:59:22.264339 8119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:22.242622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42712 (local address 127.2.237.3:34437)
0620 22:59:22.260059 (+ 17437us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:22.260067 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:22.260100 (+ 33us) server_negotiation.cc:407] Connection header received
0620 22:59:22.260188 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:22.260195 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:22.260382 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:22.261534 (+ 1152us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:22.262536 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:22.262544 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:22.262556 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 22:59:22.262593 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:22.262641 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 22:59:22.262659 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:22.263781 (+ 1122us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:22.263845 (+ 64us) server_negotiation.cc:299] Negotiation successful
0620 22:59:22.263935 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17214,"thread_start_us":17046,"threads_started":1}
I20260620 22:59:22.266944 7777 tablet_service.cc:1467] Processing CreateTablet for tablet 8b311066c13e40e5ac27d94414fd858e (DEFAULT_TABLE table=test-table [id=21f7dc4317a048aeaa52c33d3f199507]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:59:22.267446 8038 tablet_service.cc:1467] Processing CreateTablet for tablet 8b311066c13e40e5ac27d94414fd858e (DEFAULT_TABLE table=test-table [id=21f7dc4317a048aeaa52c33d3f199507]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:59:22.267593 7908 tablet_service.cc:1467] Processing CreateTablet for tablet 8b311066c13e40e5ac27d94414fd858e (DEFAULT_TABLE table=test-table [id=21f7dc4317a048aeaa52c33d3f199507]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:59:22.268481 7777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b311066c13e40e5ac27d94414fd858e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:22.268747 8038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b311066c13e40e5ac27d94414fd858e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:22.268913 7908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b311066c13e40e5ac27d94414fd858e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:22.279635 8121 tablet_bootstrap.cc:492] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4: Bootstrap starting.
I20260620 22:59:22.283759 8122 tablet_bootstrap.cc:492] T 8b311066c13e40e5ac27d94414fd858e P fae520cec6d34a4b8294d40c1871a9b9: Bootstrap starting.
I20260620 22:59:22.284938 8123 tablet_bootstrap.cc:492] T 8b311066c13e40e5ac27d94414fd858e P bd548ce251c24115a9dd265f32d923c0: Bootstrap starting.
I20260620 22:59:22.285238 8121 tablet_bootstrap.cc:654] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:22.286545 8121 log.cc:824] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4: Log is configured to *not* fsync() on all Append() calls
I20260620 22:59:22.287253 8122 tablet_bootstrap.cc:654] T 8b311066c13e40e5ac27d94414fd858e P fae520cec6d34a4b8294d40c1871a9b9: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:22.287814 8123 tablet_bootstrap.cc:654] T 8b311066c13e40e5ac27d94414fd858e P bd548ce251c24115a9dd265f32d923c0: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:22.288321 8122 log.cc:824] T 8b311066c13e40e5ac27d94414fd858e P fae520cec6d34a4b8294d40c1871a9b9: Log is configured to *not* fsync() on all Append() calls
I20260620 22:59:22.288823 8123 log.cc:824] T 8b311066c13e40e5ac27d94414fd858e P bd548ce251c24115a9dd265f32d923c0: Log is configured to *not* fsync() on all Append() calls
I20260620 22:59:22.289834 8121 tablet_bootstrap.cc:492] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4: No bootstrap required, opened a new log
I20260620 22:59:22.290107 8121 ts_tablet_manager.cc:1397] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260620 22:59:22.291453 8122 tablet_bootstrap.cc:492] T 8b311066c13e40e5ac27d94414fd858e P fae520cec6d34a4b8294d40c1871a9b9: No bootstrap required, opened a new log
I20260620 22:59:22.291718 8122 ts_tablet_manager.cc:1397] T 8b311066c13e40e5ac27d94414fd858e P fae520cec6d34a4b8294d40c1871a9b9: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.001s
I20260620 22:59:22.292330 8123 tablet_bootstrap.cc:492] T 8b311066c13e40e5ac27d94414fd858e P bd548ce251c24115a9dd265f32d923c0: No bootstrap required, opened a new log
I20260620 22:59:22.292574 8123 ts_tablet_manager.cc:1397] T 8b311066c13e40e5ac27d94414fd858e P bd548ce251c24115a9dd265f32d923c0: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260620 22:59:22.298283 8121 raft_consensus.cc:348] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae520cec6d34a4b8294d40c1871a9b9" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33013 } } peers { permanent_uuid: "38540c7404a140d9b43eb24cba8c15c4" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34437 } } peers { permanent_uuid: "bd548ce251c24115a9dd265f32d923c0" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 40815 } }
I20260620 22:59:22.298695 8121 raft_consensus.cc:374] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:22.298795 8121 raft_consensus.cc:729] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38540c7404a140d9b43eb24cba8c15c4, State: Initialized, Role: FOLLOWER
I20260620 22:59:22.299413 8121 consensus_queue.cc:260] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae520cec6d34a4b8294d40c1871a9b9" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33013 } } peers { permanent_uuid: "38540c7404a140d9b43eb24cba8c15c4" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34437 } } peers { permanent_uuid: "bd548ce251c24115a9dd265f32d923c0" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 40815 } }
I20260620 22:59:22.300513 8103 heartbeater.cc:503] Master 127.2.237.62:33373 was elected leader, sending a full tablet report...
I20260620 22:59:22.301206 8121 ts_tablet_manager.cc:1428] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4: Time spent starting tablet: real 0.011s user 0.010s sys 0.001s
I20260620 22:59:22.302588 8122 raft_consensus.cc:348] T 8b311066c13e40e5ac27d94414fd858e P fae520cec6d34a4b8294d40c1871a9b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae520cec6d34a4b8294d40c1871a9b9" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33013 } } peers { permanent_uuid: "38540c7404a140d9b43eb24cba8c15c4" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34437 } } peers { permanent_uuid: "bd548ce251c24115a9dd265f32d923c0" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 40815 } }
I20260620 22:59:22.303050 8122 raft_consensus.cc:374] T 8b311066c13e40e5ac27d94414fd858e P fae520cec6d34a4b8294d40c1871a9b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:22.303165 8122 raft_consensus.cc:729] T 8b311066c13e40e5ac27d94414fd858e P fae520cec6d34a4b8294d40c1871a9b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fae520cec6d34a4b8294d40c1871a9b9, State: Initialized, Role: FOLLOWER
I20260620 22:59:22.303028 8123 raft_consensus.cc:348] T 8b311066c13e40e5ac27d94414fd858e P bd548ce251c24115a9dd265f32d923c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae520cec6d34a4b8294d40c1871a9b9" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33013 } } peers { permanent_uuid: "38540c7404a140d9b43eb24cba8c15c4" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34437 } } peers { permanent_uuid: "bd548ce251c24115a9dd265f32d923c0" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 40815 } }
I20260620 22:59:22.303463 8123 raft_consensus.cc:374] T 8b311066c13e40e5ac27d94414fd858e P bd548ce251c24115a9dd265f32d923c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:22.303567 8123 raft_consensus.cc:729] T 8b311066c13e40e5ac27d94414fd858e P bd548ce251c24115a9dd265f32d923c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd548ce251c24115a9dd265f32d923c0, State: Initialized, Role: FOLLOWER
I20260620 22:59:22.303872 8122 consensus_queue.cc:260] T 8b311066c13e40e5ac27d94414fd858e P fae520cec6d34a4b8294d40c1871a9b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae520cec6d34a4b8294d40c1871a9b9" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33013 } } peers { permanent_uuid: "38540c7404a140d9b43eb24cba8c15c4" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34437 } } peers { permanent_uuid: "bd548ce251c24115a9dd265f32d923c0" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 40815 } }
I20260620 22:59:22.304131 8123 consensus_queue.cc:260] T 8b311066c13e40e5ac27d94414fd858e P bd548ce251c24115a9dd265f32d923c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae520cec6d34a4b8294d40c1871a9b9" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33013 } } peers { permanent_uuid: "38540c7404a140d9b43eb24cba8c15c4" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34437 } } peers { permanent_uuid: "bd548ce251c24115a9dd265f32d923c0" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 40815 } }
I20260620 22:59:22.305230 7973 heartbeater.cc:503] Master 127.2.237.62:33373 was elected leader, sending a full tablet report...
I20260620 22:59:22.306211 8123 ts_tablet_manager.cc:1428] T 8b311066c13e40e5ac27d94414fd858e P bd548ce251c24115a9dd265f32d923c0: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260620 22:59:22.307785 7842 heartbeater.cc:503] Master 127.2.237.62:33373 was elected leader, sending a full tablet report...
I20260620 22:59:22.308322 8122 ts_tablet_manager.cc:1428] T 8b311066c13e40e5ac27d94414fd858e P fae520cec6d34a4b8294d40c1871a9b9: Time spent starting tablet: real 0.016s user 0.010s sys 0.006s
W20260620 22:59:22.333334 7843 tablet.cc:2369] T 8b311066c13e40e5ac27d94414fd858e P fae520cec6d34a4b8294d40c1871a9b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 22:59:22.348452 8127 raft_consensus.cc:482] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 22:59:22.348789 8127 raft_consensus.cc:504] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae520cec6d34a4b8294d40c1871a9b9" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33013 } } peers { permanent_uuid: "38540c7404a140d9b43eb24cba8c15c4" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34437 } } peers { permanent_uuid: "bd548ce251c24115a9dd265f32d923c0" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 40815 } }
I20260620 22:59:22.350660 8127 leader_election.cc:288] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fae520cec6d34a4b8294d40c1871a9b9 (127.2.237.1:33013), bd548ce251c24115a9dd265f32d923c0 (127.2.237.2:40815)
I20260620 22:59:22.353103 8117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:22.350957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:34049 (local address 127.2.237.1:33013)
0620 22:59:22.351136 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:22.351144 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:22.351168 (+ 24us) server_negotiation.cc:407] Connection header received
0620 22:59:22.351347 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:22.351353 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:22.351417 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:22.351537 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:22.352483 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:22.352493 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:22.352496 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 22:59:22.352511 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:22.352539 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 22:59:22.352549 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:22.352663 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:22.352807 (+ 144us) server_negotiation.cc:299] Negotiation successful
0620 22:59:22.352861 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260620 22:59:22.353099 8105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:22.350831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:33013 (local address 127.2.237.3:34049)
0620 22:59:22.351047 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:22.351067 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:22.351172 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:22.351580 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:22.351585 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:22.351608 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:22.352289 (+ 681us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 22:59:22.352291 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:22.352305 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 22:59:22.352343 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:22.352639 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:22.352643 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:22.352648 (+ 5us) client_negotiation.cc:769] Sending connection context
0620 22:59:22.352800 (+ 152us) client_negotiation.cc:240] Negotiation successful
0620 22:59:22.352853 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260620 22:59:22.353552 8120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:22.351204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:45061 (local address 127.2.237.2:40815)
0620 22:59:22.351373 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:22.351378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:22.351743 (+ 365us) server_negotiation.cc:407] Connection header received
0620 22:59:22.351845 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:22.351852 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:22.351919 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:22.351994 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:22.352942 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:22.352947 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:22.352954 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 22:59:22.352974 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:22.353000 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 22:59:22.353012 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:22.353138 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:22.353271 (+ 133us) server_negotiation.cc:299] Negotiation successful
0620 22:59:22.353337 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260620 22:59:22.354074 8130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:22.351042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:40815 (local address 127.2.237.3:45061)
0620 22:59:22.351636 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:22.351653 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:22.351742 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:22.352053 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:22.352061 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:22.352087 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:22.352754 (+ 667us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 22:59:22.352756 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:22.352768 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 22:59:22.352802 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:22.353109 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:22.353113 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:22.353119 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 22:59:22.353765 (+ 646us) client_negotiation.cc:240] Negotiation successful
0620 22:59:22.353804 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":310,"threads_started":1}
I20260620 22:59:22.353992 7797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b311066c13e40e5ac27d94414fd858e" candidate_uuid: "38540c7404a140d9b43eb24cba8c15c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae520cec6d34a4b8294d40c1871a9b9" is_pre_election: true
I20260620 22:59:22.354487 7797 raft_consensus.cc:2454] T 8b311066c13e40e5ac27d94414fd858e P fae520cec6d34a4b8294d40c1871a9b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38540c7404a140d9b43eb24cba8c15c4 in term 0.
I20260620 22:59:22.355172 7994 leader_election.cc:302] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38540c7404a140d9b43eb24cba8c15c4, fae520cec6d34a4b8294d40c1871a9b9; no voters:
I20260620 22:59:22.355547 8127 raft_consensus.cc:2781] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 22:59:22.355373 7928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b311066c13e40e5ac27d94414fd858e" candidate_uuid: "38540c7404a140d9b43eb24cba8c15c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd548ce251c24115a9dd265f32d923c0" is_pre_election: true
I20260620 22:59:22.355692 8127 raft_consensus.cc:482] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 22:59:22.355805 8127 raft_consensus.cc:3037] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:22.355913 7928 raft_consensus.cc:2454] T 8b311066c13e40e5ac27d94414fd858e P bd548ce251c24115a9dd265f32d923c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38540c7404a140d9b43eb24cba8c15c4 in term 0.
I20260620 22:59:22.358564 8127 raft_consensus.cc:504] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae520cec6d34a4b8294d40c1871a9b9" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33013 } } peers { permanent_uuid: "38540c7404a140d9b43eb24cba8c15c4" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34437 } } peers { permanent_uuid: "bd548ce251c24115a9dd265f32d923c0" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 40815 } }
I20260620 22:59:22.359171 8127 leader_election.cc:288] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4 [CANDIDATE]: Term 1 election: Requested vote from peers fae520cec6d34a4b8294d40c1871a9b9 (127.2.237.1:33013), bd548ce251c24115a9dd265f32d923c0 (127.2.237.2:40815)
I20260620 22:59:22.359535 7797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b311066c13e40e5ac27d94414fd858e" candidate_uuid: "38540c7404a140d9b43eb24cba8c15c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae520cec6d34a4b8294d40c1871a9b9"
I20260620 22:59:22.359598 7928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b311066c13e40e5ac27d94414fd858e" candidate_uuid: "38540c7404a140d9b43eb24cba8c15c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd548ce251c24115a9dd265f32d923c0"
I20260620 22:59:22.359738 7797 raft_consensus.cc:3037] T 8b311066c13e40e5ac27d94414fd858e P fae520cec6d34a4b8294d40c1871a9b9 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:22.359815 7928 raft_consensus.cc:3037] T 8b311066c13e40e5ac27d94414fd858e P bd548ce251c24115a9dd265f32d923c0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:22.362166 7928 raft_consensus.cc:2454] T 8b311066c13e40e5ac27d94414fd858e P bd548ce251c24115a9dd265f32d923c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38540c7404a140d9b43eb24cba8c15c4 in term 1.
I20260620 22:59:22.362191 7797 raft_consensus.cc:2454] T 8b311066c13e40e5ac27d94414fd858e P fae520cec6d34a4b8294d40c1871a9b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38540c7404a140d9b43eb24cba8c15c4 in term 1.
I20260620 22:59:22.362540 7993 leader_election.cc:302] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38540c7404a140d9b43eb24cba8c15c4, bd548ce251c24115a9dd265f32d923c0; no voters:
I20260620 22:59:22.362912 8127 raft_consensus.cc:2781] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:59:22.363374 8127 raft_consensus.cc:686] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4 [term 1 LEADER]: Becoming Leader. State: Replica: 38540c7404a140d9b43eb24cba8c15c4, State: Running, Role: LEADER
I20260620 22:59:22.363893 8127 consensus_queue.cc:237] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae520cec6d34a4b8294d40c1871a9b9" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33013 } } peers { permanent_uuid: "38540c7404a140d9b43eb24cba8c15c4" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34437 } } peers { permanent_uuid: "bd548ce251c24115a9dd265f32d923c0" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 40815 } }
I20260620 22:59:22.368530 7656 catalog_manager.cc:5495] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 38540c7404a140d9b43eb24cba8c15c4 (127.2.237.3). New cstate: current_term: 1 leader_uuid: "38540c7404a140d9b43eb24cba8c15c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae520cec6d34a4b8294d40c1871a9b9" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38540c7404a140d9b43eb24cba8c15c4" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd548ce251c24115a9dd265f32d923c0" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 40815 } health_report { overall_health: UNKNOWN } } }
I20260620 22:59:22.406373 8119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:22.404107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42720 (local address 127.2.237.3:34437)
0620 22:59:22.404314 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:22.404321 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:22.404428 (+ 107us) server_negotiation.cc:407] Connection header received
0620 22:59:22.404693 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:22.404698 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:22.404782 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:22.404896 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:22.405769 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:22.405775 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:22.405779 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 22:59:22.405797 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:22.405835 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 22:59:22.405847 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:22.405976 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:22.406059 (+ 83us) server_negotiation.cc:299] Negotiation successful
0620 22:59:22.406116 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260620 22:59:22.414418 7928 raft_consensus.cc:1261] T 8b311066c13e40e5ac27d94414fd858e P bd548ce251c24115a9dd265f32d923c0 [term 1 FOLLOWER]: Refusing update from remote peer 38540c7404a140d9b43eb24cba8c15c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:59:22.414424 7797 raft_consensus.cc:1261] T 8b311066c13e40e5ac27d94414fd858e P fae520cec6d34a4b8294d40c1871a9b9 [term 1 FOLLOWER]: Refusing update from remote peer 38540c7404a140d9b43eb24cba8c15c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:59:22.415475 8131 consensus_queue.cc:1035] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fae520cec6d34a4b8294d40c1871a9b9" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:59:22.415799 8127 consensus_queue.cc:1035] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd548ce251c24115a9dd265f32d923c0" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 40815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:59:22.429546 8134 mvcc.cc:205] Tried to move back new op lower bound from 7299057100439523328 to 7299057100246720512. Current Snapshot: MvccSnapshot[applied={T|T < 7299057100439523328}]
I20260620 22:59:22.443038 7655 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 22:59:22.443279 7655 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 22:59:22.446748 7655 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9ba8e08954a2456ca9032a7f9a137e3b: Sending DeleteTablet for 3 replicas of tablet 8b311066c13e40e5ac27d94414fd858e
I20260620 22:59:22.447800 7777 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b311066c13e40e5ac27d94414fd858e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:22 UTC) from {username='slave'} at 127.0.0.1:38572
I20260620 22:59:22.447827 8038 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b311066c13e40e5ac27d94414fd858e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:22 UTC) from {username='slave'} at 127.0.0.1:42712
I20260620 22:59:22.447815 7908 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b311066c13e40e5ac27d94414fd858e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:22 UTC) from {username='slave'} at 127.0.0.1:59748
I20260620 22:59:22.452181 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 7715
I20260620 22:59:22.460958 8146 tablet_replica.cc:330] T 8b311066c13e40e5ac27d94414fd858e P bd548ce251c24115a9dd265f32d923c0: stopping tablet replica
I20260620 22:59:22.461557 8144 tablet_replica.cc:330] T 8b311066c13e40e5ac27d94414fd858e P fae520cec6d34a4b8294d40c1871a9b9: stopping tablet replica
I20260620 22:59:22.462218 8144 raft_consensus.cc:2229] T 8b311066c13e40e5ac27d94414fd858e P fae520cec6d34a4b8294d40c1871a9b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 22:59:22.462812 8146 raft_consensus.cc:2229] T 8b311066c13e40e5ac27d94414fd858e P bd548ce251c24115a9dd265f32d923c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 22:59:22.463445 8144 raft_consensus.cc:2258] T 8b311066c13e40e5ac27d94414fd858e P fae520cec6d34a4b8294d40c1871a9b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:22.463976 8146 raft_consensus.cc:2258] T 8b311066c13e40e5ac27d94414fd858e P bd548ce251c24115a9dd265f32d923c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:22.465478 8145 tablet_replica.cc:330] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4: stopping tablet replica
I20260620 22:59:22.467068 8146 ts_tablet_manager.cc:1905] T 8b311066c13e40e5ac27d94414fd858e P bd548ce251c24115a9dd265f32d923c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:22.467905 8144 ts_tablet_manager.cc:1905] T 8b311066c13e40e5ac27d94414fd858e P fae520cec6d34a4b8294d40c1871a9b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:22.469766 8117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:22.454012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38578 (local address 127.2.237.1:33013)
0620 22:59:22.454254 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:22.454264 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:22.467015 (+ 12751us) server_negotiation.cc:407] Connection header received
0620 22:59:22.467539 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:22.467549 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:22.467635 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:22.467717 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:22.468954 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:22.468960 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:22.468966 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 22:59:22.468990 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:22.469032 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 22:59:22.469046 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:22.469143 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:22.469380 (+ 237us) server_negotiation.cc:299] Negotiation successful
0620 22:59:22.469451 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260620 22:59:22.470829 7837 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:22.473336 8145 raft_consensus.cc:2229] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4 [term 1 LEADER]: Raft consensus shutting down.
I20260620 22:59:22.474179 8145 raft_consensus.cc:2258] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:22.475127 8146 ts_tablet_manager.cc:1918] T 8b311066c13e40e5ac27d94414fd858e P bd548ce251c24115a9dd265f32d923c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 22:59:22.475359 8146 log.cc:1196] T 8b311066c13e40e5ac27d94414fd858e P bd548ce251c24115a9dd265f32d923c0: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-1/wal/wals/8b311066c13e40e5ac27d94414fd858e
I20260620 22:59:22.475963 8146 ts_tablet_manager.cc:1939] T 8b311066c13e40e5ac27d94414fd858e P bd548ce251c24115a9dd265f32d923c0: Deleting consensus metadata
I20260620 22:59:22.476848 8145 ts_tablet_manager.cc:1905] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:22.477984 7642 catalog_manager.cc:4841] TS bd548ce251c24115a9dd265f32d923c0 (127.2.237.2:40815): tablet 8b311066c13e40e5ac27d94414fd858e (table test-table [id=21f7dc4317a048aeaa52c33d3f199507]) successfully deleted
I20260620 22:59:22.484261 8145 ts_tablet_manager.cc:1918] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 22:59:22.484460 8145 log.cc:1196] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781996318807807-2996-0/minicluster-data/ts-2/wal/wals/8b311066c13e40e5ac27d94414fd858e
I20260620 22:59:22.484944 8145 ts_tablet_manager.cc:1939] T 8b311066c13e40e5ac27d94414fd858e P 38540c7404a140d9b43eb24cba8c15c4: Deleting consensus metadata
I20260620 22:59:22.486495 7641 catalog_manager.cc:4841] TS 38540c7404a140d9b43eb24cba8c15c4 (127.2.237.3:34437): tablet 8b311066c13e40e5ac27d94414fd858e (table test-table [id=21f7dc4317a048aeaa52c33d3f199507]) successfully deleted
I20260620 22:59:22.579011 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 7715
W20260620 22:59:22.601890 7643 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.237.1:33013 (error 108)
I20260620 22:59:22.602721 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 7846
I20260620 22:59:22.604032 8116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:22.603541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:33013 (local address 127.0.0.1:38582)
0620 22:59:22.603744 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:22.603861 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:33013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260620 22:59:22.604233 7643 catalog_manager.cc:4584] TS fae520cec6d34a4b8294d40c1871a9b9 (127.2.237.1:33013): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8b311066c13e40e5ac27d94414fd858e: Network error: Client connection negotiation failed: client connection to 127.2.237.1:33013: connect: Connection refused (error 111)
I20260620 22:59:22.606518 8120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:22.603574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59764 (local address 127.2.237.2:40815)
0620 22:59:22.603750 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:22.603757 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:22.604599 (+ 842us) server_negotiation.cc:407] Connection header received
0620 22:59:22.604723 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:22.604729 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:22.604854 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:22.604964 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:22.605865 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:22.605871 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:22.605876 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 22:59:22.605896 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:22.605935 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 22:59:22.605951 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:22.606084 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:22.606174 (+ 90us) server_negotiation.cc:299] Negotiation successful
0620 22:59:22.606233 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 22:59:22.607367 7968 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:22.657972 8116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:22.657364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:33013 (local address 127.0.0.1:38590)
0620 22:59:22.657615 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:22.657741 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:33013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260620 22:59:22.659132 8116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:22.658734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:33013 (local address 127.0.0.1:38606)
0620 22:59:22.658892 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:22.658977 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:33013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260620 22:59:22.696509 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 7846
I20260620 22:59:22.719695 8116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:22.719154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:33013 (local address 127.0.0.1:38614)
0620 22:59:22.719377 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:22.719454 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:33013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260620 22:59:22.719723 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 7976
I20260620 22:59:22.720754 8116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:22.720393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:33013 (local address 127.0.0.1:38624)
0620 22:59:22.720532 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:22.720620 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:33013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260620 22:59:22.723338 8119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:22.720515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42726 (local address 127.2.237.3:34437)
0620 22:59:22.721029 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:22.721034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:22.721054 (+ 20us) server_negotiation.cc:407] Connection header received
0620 22:59:22.721259 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:22.721264 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:22.721314 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:22.721394 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:22.722479 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:22.722487 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:22.722491 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 22:59:22.722505 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:22.722532 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 22:59:22.722546 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:22.722880 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:22.722981 (+ 101us) server_negotiation.cc:299] Negotiation successful
0620 22:59:22.723033 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366}
I20260620 22:59:22.724064 8098 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:22.801407 8116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:22.800866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:33013 (local address 127.0.0.1:38628)
0620 22:59:22.801104 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:22.801228 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:33013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260620 22:59:22.802490 8116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:22.802102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:33013 (local address 127.0.0.1:38642)
0620 22:59:22.802314 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:22.802376 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:33013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260620 22:59:22.847646 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 7976
I20260620 22:59:22.866468 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 7625
I20260620 22:59:22.867636 7686 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:22.972743 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 7625
2026-06-20T22:59:22Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2651 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260620 22:59:23.015240 2996 mini_oidc.cc:152] Starting JWKS server
I20260620 22:59:23.017880 2996 webserver.cc:460] Webserver started at https://127.0.0.1:33839/ using document root <none> and password file <none>
I20260620 22:59:23.018167 2996 mini_oidc.cc:189] Starting OIDC Discovery server
I20260620 22:59:23.018854 2996 webserver.cc:460] Webserver started at http://127.0.0.1:44411/ using document root <none> and password file <none>
2026-06-20T22:59:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T22:59:23Z Disabled control of system clock
I20260620 22:59:23.046675 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:38861
--webserver_interface=127.2.237.62
--webserver_port=0
--builtin_ntp_servers=127.2.237.20:43325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:38861
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33839/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996318807807-2996-0/testchainca.cert with env {}
W20260620 22:59:23.371564 8158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:23.371878 8158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:23.371968 8158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:23.379189 8158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 22:59:23.379297 8158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:23.379333 8158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:23.379390 8158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 22:59:23.379434 8158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 22:59:23.379487 8158 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 22:59:23.379526 8158 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33839/jwks.json
I20260620 22:59:23.389714 8158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:43325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:38861
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:38861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33839/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996318807807-2996-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:23.391134 8158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:23.392390 8158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:23.404318 8164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:23.404390 8166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:23.404390 8163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:23.405257 8158 server_base.cc:1034] running on GCE node
I20260620 22:59:23.406032 8158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:23.407460 8158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:23.408707 8158 hybrid_clock.cc:648] HybridClock initialized: now 1781996363408645 us; error 53 us; skew 500 ppm
I20260620 22:59:23.409085 8158 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 22:59:23.411381 8158 webserver.cc:460] Webserver started at http://127.2.237.62:42449/ using document root <none> and password file <none>
I20260620 22:59:23.412072 8158 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:23.412187 8158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:23.412508 8158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:23.415535 8158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "c10799f10e514b62b26091b4b0c86e75"
format_stamp: "Formatted at 2026-06-20 22:59:23 on dist-test-slave-f63b"
server_key: "a6af4e893954adb181e2bcbbe336fc46"
server_key_iv: "ed6214b23cfa369825fc0af646d678c9"
server_key_version: "encryptionkey@0"
I20260620 22:59:23.416328 8158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "c10799f10e514b62b26091b4b0c86e75"
format_stamp: "Formatted at 2026-06-20 22:59:23 on dist-test-slave-f63b"
server_key: "a6af4e893954adb181e2bcbbe336fc46"
server_key_iv: "ed6214b23cfa369825fc0af646d678c9"
server_key_version: "encryptionkey@0"
I20260620 22:59:23.421273 8158 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260620 22:59:23.424778 8172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:23.426143 8158 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 22:59:23.426311 8158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "c10799f10e514b62b26091b4b0c86e75"
format_stamp: "Formatted at 2026-06-20 22:59:23 on dist-test-slave-f63b"
server_key: "a6af4e893954adb181e2bcbbe336fc46"
server_key_iv: "ed6214b23cfa369825fc0af646d678c9"
server_key_version: "encryptionkey@0"
I20260620 22:59:23.426522 8158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:23.461534 8158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:23.462546 8158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:23.462904 8158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:23.481058 8158 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:38861
I20260620 22:59:23.481069 8223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:38861 every 8 connection(s)
I20260620 22:59:23.482463 8158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 22:59:23.485376 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 8158
I20260620 22:59:23.485644 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 22:59:23.486248 2996 external_mini_cluster.cc:1428] Setting key 8c8564a3137e879babc89691c91cd66c
I20260620 22:59:23.486325 8224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:23.498816 8224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c10799f10e514b62b26091b4b0c86e75: Bootstrap starting.
I20260620 22:59:23.499790 8227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:23.488174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37298 (local address 127.2.237.62:38861)
0620 22:59:23.489226 (+ 1052us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:23.489236 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:23.489282 (+ 46us) server_negotiation.cc:407] Connection header received
0620 22:59:23.491181 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:23.491203 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:23.491670 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:23.492102 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:23.492697 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:23.493972 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:23.495390 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:23.496059 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:23.496297 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:23.497502 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:23.497510 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:23.497525 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 22:59:23.498035 (+ 510us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:23.498116 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 22:59:23.498136 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:23.498270 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:23.498452 (+ 182us) server_negotiation.cc:299] Negotiation successful
0620 22:59:23.498802 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":332,"threads_started":1}
I20260620 22:59:23.502466 8224 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c10799f10e514b62b26091b4b0c86e75: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:23.503585 8224 log.cc:824] T 00000000000000000000000000000000 P c10799f10e514b62b26091b4b0c86e75: Log is configured to *not* fsync() on all Append() calls
I20260620 22:59:23.503933 2996 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 22:59:23.506302 8224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c10799f10e514b62b26091b4b0c86e75: No bootstrap required, opened a new log
I20260620 22:59:23.513517 8224 raft_consensus.cc:348] T 00000000000000000000000000000000 P c10799f10e514b62b26091b4b0c86e75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10799f10e514b62b26091b4b0c86e75" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 38861 } }
I20260620 22:59:23.513886 8224 raft_consensus.cc:374] T 00000000000000000000000000000000 P c10799f10e514b62b26091b4b0c86e75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:23.513983 8224 raft_consensus.cc:729] T 00000000000000000000000000000000 P c10799f10e514b62b26091b4b0c86e75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c10799f10e514b62b26091b4b0c86e75, State: Initialized, Role: FOLLOWER
I20260620 22:59:23.514637 8224 consensus_queue.cc:260] T 00000000000000000000000000000000 P c10799f10e514b62b26091b4b0c86e75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10799f10e514b62b26091b4b0c86e75" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 38861 } }
I20260620 22:59:23.514878 8224 raft_consensus.cc:388] T 00000000000000000000000000000000 P c10799f10e514b62b26091b4b0c86e75 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 22:59:23.514998 8224 raft_consensus.cc:482] T 00000000000000000000000000000000 P c10799f10e514b62b26091b4b0c86e75 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 22:59:23.515151 8224 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c10799f10e514b62b26091b4b0c86e75 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:23.517454 8224 raft_consensus.cc:504] T 00000000000000000000000000000000 P c10799f10e514b62b26091b4b0c86e75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10799f10e514b62b26091b4b0c86e75" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 38861 } }
I20260620 22:59:23.517993 8224 leader_election.cc:302] T 00000000000000000000000000000000 P c10799f10e514b62b26091b4b0c86e75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c10799f10e514b62b26091b4b0c86e75; no voters:
I20260620 22:59:23.518656 8224 leader_election.cc:288] T 00000000000000000000000000000000 P c10799f10e514b62b26091b4b0c86e75 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 22:59:23.518854 8229 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c10799f10e514b62b26091b4b0c86e75 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:59:23.520284 8229 raft_consensus.cc:686] T 00000000000000000000000000000000 P c10799f10e514b62b26091b4b0c86e75 [term 1 LEADER]: Becoming Leader. State: Replica: c10799f10e514b62b26091b4b0c86e75, State: Running, Role: LEADER
I20260620 22:59:23.520833 8229 consensus_queue.cc:237] T 00000000000000000000000000000000 P c10799f10e514b62b26091b4b0c86e75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10799f10e514b62b26091b4b0c86e75" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 38861 } }
I20260620 22:59:23.521062 8224 sys_catalog.cc:564] T 00000000000000000000000000000000 P c10799f10e514b62b26091b4b0c86e75 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 22:59:23.523892 8231 sys_catalog.cc:455] T 00000000000000000000000000000000 P c10799f10e514b62b26091b4b0c86e75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c10799f10e514b62b26091b4b0c86e75. Latest consensus state: current_term: 1 leader_uuid: "c10799f10e514b62b26091b4b0c86e75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10799f10e514b62b26091b4b0c86e75" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 38861 } } }
I20260620 22:59:23.523999 8230 sys_catalog.cc:455] T 00000000000000000000000000000000 P c10799f10e514b62b26091b4b0c86e75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c10799f10e514b62b26091b4b0c86e75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c10799f10e514b62b26091b4b0c86e75" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 38861 } } }
I20260620 22:59:23.524312 8231 sys_catalog.cc:458] T 00000000000000000000000000000000 P c10799f10e514b62b26091b4b0c86e75 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:23.524369 8230 sys_catalog.cc:458] T 00000000000000000000000000000000 P c10799f10e514b62b26091b4b0c86e75 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:23.527550 8237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 22:59:23.532869 8237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 22:59:23.542029 8237 catalog_manager.cc:1347] Generated new cluster ID: 9cf0687451f74bd5afc624b6ad94bd33
I20260620 22:59:23.542164 8237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 22:59:23.560156 8237 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 22:59:23.561527 8237 catalog_manager.cc:1504] Loading token signing keys...
I20260620 22:59:23.571852 8237 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c10799f10e514b62b26091b4b0c86e75: Generated new TSK 0
I20260620 22:59:23.572700 8237 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260620 22:59:23.608685 8254 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260620 22:59:23.608870 8254 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260620 22:59:23.609799 8227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:23.588960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37302 (local address 127.2.237.62:38861)
0620 22:59:23.589181 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:23.589190 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:23.589614 (+ 424us) server_negotiation.cc:407] Connection header received
0620 22:59:23.589962 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:23.589966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:23.590032 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:23.590155 (+ 123us) server_negotiation.cc:226] Negotiated authn=JWT
0620 22:59:23.590804 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:23.591547 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:23.593570 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:23.594006 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:23.594096 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:23.594791 (+ 695us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 22:59:23.609297 (+ 14506us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33839/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0620 22:59:23.609566 (+ 269us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37302: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33839/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":53}
W20260620 22:59:23.609949 8227 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37302: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33839/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260620 22:59:23.611845 8253 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 22:59:23.588751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:38861 (local address 127.0.0.1:37302)
0620 22:59:23.589492 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:23.589513 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:23.589727 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:23.590222 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:23.590227 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:23.590241 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0620 22:59:23.590630 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:23.590642 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:23.591693 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:23.591699 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:23.593430 (+ 1731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:23.593445 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:23.594167 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:23.594175 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:23.594373 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:23.594449 (+ 76us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 22:59:23.609665 (+ 15216us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33839/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0620 22:59:23.610026 (+ 361us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.237.62:38861: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33839/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":359,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.2.237.62:38861: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33839/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260620 22:59:23.613409 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 8158
I20260620 22:59:23.614614 8219 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:23.707142 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 8158
2026-06-20T22:59:23Z chronyd exiting
I20260620 22:59:23.757923 2996 test_util.cc:175] -----------------------------------------------
I20260620 22:59:23.758108 2996 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781996318807807-2996-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (747 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260620 22:59:23.762253 2996 mini_oidc.cc:152] Starting JWKS server
I20260620 22:59:23.764372 2996 webserver.cc:460] Webserver started at https://127.0.0.1:40865/ using document root <none> and password file <none>
I20260620 22:59:23.764643 2996 mini_oidc.cc:189] Starting OIDC Discovery server
I20260620 22:59:23.765354 2996 webserver.cc:460] Webserver started at http://127.0.0.1:41719/ using document root <none> and password file <none>
2026-06-20T22:59:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T22:59:23Z Disabled control of system clock
I20260620 22:59:23.792310 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:40387
--webserver_interface=127.2.237.62
--webserver_port=0
--builtin_ntp_servers=127.2.237.20:39983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:40387
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40865/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996318807807-2996-0/testchainca.cert with env {}
W20260620 22:59:24.110668 8261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:24.110954 8261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:24.111040 8261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:24.118075 8261 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 22:59:24.118183 8261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:24.118228 8261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:24.118278 8261 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 22:59:24.118320 8261 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 22:59:24.118373 8261 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 22:59:24.118412 8261 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40865/jwks.json
I20260620 22:59:24.128204 8261 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:39983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:40387
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:40387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40865/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996318807807-2996-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:24.129473 8261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:24.130860 8261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:24.142639 8266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:24.142541 8267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:24.142565 8269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:24.143661 8261 server_base.cc:1034] running on GCE node
I20260620 22:59:24.144418 8261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:24.145771 8261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:24.146960 8261 hybrid_clock.cc:648] HybridClock initialized: now 1781996364146895 us; error 56 us; skew 500 ppm
I20260620 22:59:24.147287 8261 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 22:59:24.153371 8261 webserver.cc:460] Webserver started at http://127.2.237.62:36011/ using document root <none> and password file <none>
I20260620 22:59:24.154048 8261 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:24.154146 8261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:24.154543 8261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:24.157459 8261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "b8c25806195a419a84b03f6699ed3be3"
format_stamp: "Formatted at 2026-06-20 22:59:24 on dist-test-slave-f63b"
server_key: "f7ea969453d7bb9165cddb4dd6ec8d6d"
server_key_iv: "e3b09e02cb6cc7c6306e30d093ae4548"
server_key_version: "encryptionkey@0"
I20260620 22:59:24.158787 8261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "b8c25806195a419a84b03f6699ed3be3"
format_stamp: "Formatted at 2026-06-20 22:59:24 on dist-test-slave-f63b"
server_key: "f7ea969453d7bb9165cddb4dd6ec8d6d"
server_key_iv: "e3b09e02cb6cc7c6306e30d093ae4548"
server_key_version: "encryptionkey@0"
I20260620 22:59:24.163640 8261 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260620 22:59:24.166949 8275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:24.168239 8261 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260620 22:59:24.168426 8261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "b8c25806195a419a84b03f6699ed3be3"
format_stamp: "Formatted at 2026-06-20 22:59:24 on dist-test-slave-f63b"
server_key: "f7ea969453d7bb9165cddb4dd6ec8d6d"
server_key_iv: "e3b09e02cb6cc7c6306e30d093ae4548"
server_key_version: "encryptionkey@0"
I20260620 22:59:24.168627 8261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:24.193794 8261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:24.194664 8261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:24.194975 8261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:24.212618 8261 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:40387
I20260620 22:59:24.212625 8326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:40387 every 8 connection(s)
I20260620 22:59:24.214030 8261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 22:59:24.217680 8327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:24.221582 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 8261
I20260620 22:59:24.221786 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 22:59:24.222597 2996 external_mini_cluster.cc:1428] Setting key ddc0bcbe79fd91bb4fe7f167fcc6a747
I20260620 22:59:24.230368 8327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8c25806195a419a84b03f6699ed3be3: Bootstrap starting.
I20260620 22:59:24.233393 8327 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8c25806195a419a84b03f6699ed3be3: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:24.234505 8327 log.cc:824] T 00000000000000000000000000000000 P b8c25806195a419a84b03f6699ed3be3: Log is configured to *not* fsync() on all Append() calls
I20260620 22:59:24.235369 8330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:24.224475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52516 (local address 127.2.237.62:40387)
0620 22:59:24.225389 (+ 914us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:24.225400 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:24.225439 (+ 39us) server_negotiation.cc:407] Connection header received
0620 22:59:24.226998 (+ 1559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:24.227020 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:24.227474 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:24.227861 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:24.228434 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:24.229658 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:24.231037 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:24.231691 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:24.231897 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:24.232992 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:24.233000 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:24.233014 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 22:59:24.233577 (+ 563us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:24.233683 (+ 106us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 22:59:24.233706 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:24.233845 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:24.234085 (+ 240us) server_negotiation.cc:299] Negotiation successful
0620 22:59:24.234374 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":348,"threads_started":1}
I20260620 22:59:24.237259 8327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8c25806195a419a84b03f6699ed3be3: No bootstrap required, opened a new log
I20260620 22:59:24.238948 2996 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 22:59:24.247146 8327 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8c25806195a419a84b03f6699ed3be3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c25806195a419a84b03f6699ed3be3" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 40387 } }
I20260620 22:59:24.247550 8327 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8c25806195a419a84b03f6699ed3be3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:24.247671 8327 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8c25806195a419a84b03f6699ed3be3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c25806195a419a84b03f6699ed3be3, State: Initialized, Role: FOLLOWER
I20260620 22:59:24.248405 8327 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8c25806195a419a84b03f6699ed3be3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c25806195a419a84b03f6699ed3be3" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 40387 } }
I20260620 22:59:24.248697 8327 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8c25806195a419a84b03f6699ed3be3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 22:59:24.248848 8327 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8c25806195a419a84b03f6699ed3be3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 22:59:24.249023 8327 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8c25806195a419a84b03f6699ed3be3 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:24.252095 8327 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8c25806195a419a84b03f6699ed3be3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c25806195a419a84b03f6699ed3be3" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 40387 } }
I20260620 22:59:24.252691 8327 leader_election.cc:302] T 00000000000000000000000000000000 P b8c25806195a419a84b03f6699ed3be3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8c25806195a419a84b03f6699ed3be3; no voters:
I20260620 22:59:24.253201 8327 leader_election.cc:288] T 00000000000000000000000000000000 P b8c25806195a419a84b03f6699ed3be3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 22:59:24.253363 8332 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8c25806195a419a84b03f6699ed3be3 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:59:24.254043 8332 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8c25806195a419a84b03f6699ed3be3 [term 1 LEADER]: Becoming Leader. State: Replica: b8c25806195a419a84b03f6699ed3be3, State: Running, Role: LEADER
I20260620 22:59:24.254635 8332 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8c25806195a419a84b03f6699ed3be3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c25806195a419a84b03f6699ed3be3" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 40387 } }
I20260620 22:59:24.255390 8327 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8c25806195a419a84b03f6699ed3be3 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 22:59:24.257129 8334 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8c25806195a419a84b03f6699ed3be3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8c25806195a419a84b03f6699ed3be3. Latest consensus state: current_term: 1 leader_uuid: "b8c25806195a419a84b03f6699ed3be3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c25806195a419a84b03f6699ed3be3" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 40387 } } }
I20260620 22:59:24.257499 8334 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8c25806195a419a84b03f6699ed3be3 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:24.257988 8333 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8c25806195a419a84b03f6699ed3be3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8c25806195a419a84b03f6699ed3be3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c25806195a419a84b03f6699ed3be3" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 40387 } } }
I20260620 22:59:24.258381 8333 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8c25806195a419a84b03f6699ed3be3 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:24.261276 8338 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 22:59:24.268950 8338 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 22:59:24.277757 8338 catalog_manager.cc:1347] Generated new cluster ID: 4b819e21eed44af2bfae4fe9cd5045a8
I20260620 22:59:24.277878 8338 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 22:59:24.310982 8338 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 22:59:24.312325 8338 catalog_manager.cc:1504] Loading token signing keys...
I20260620 22:59:24.317651 8338 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8c25806195a419a84b03f6699ed3be3: Generated new TSK 0
I20260620 22:59:24.318506 8338 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260620 22:59:24.358934 8358 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260620 22:59:24.359086 8358 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260620 22:59:24.360230 8330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:24.332525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52530 (local address 127.2.237.62:40387)
0620 22:59:24.332710 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:24.332717 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:24.333295 (+ 578us) server_negotiation.cc:407] Connection header received
0620 22:59:24.333502 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:24.333507 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:24.333597 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:24.333761 (+ 164us) server_negotiation.cc:226] Negotiated authn=JWT
0620 22:59:24.334160 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:24.335343 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:24.336796 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:24.337272 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:24.337435 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:24.337922 (+ 487us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 22:59:24.359490 (+ 21568us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40865/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0620 22:59:24.359918 (+ 428us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52530: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40865/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":63}
W20260620 22:59:24.360373 8330 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52530: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40865/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260620 22:59:24.360462 8357 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 22:59:24.332526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:40387 (local address 127.0.0.1:52530)
0620 22:59:24.333188 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:24.333209 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:24.333335 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:24.333781 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:24.333785 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:24.333795 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0620 22:59:24.334017 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:24.334025 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:24.335469 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:24.335473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:24.336607 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:24.336618 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:24.337472 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:24.337479 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:24.337630 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:24.337676 (+ 46us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 22:59:24.359763 (+ 22087us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40865/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0620 22:59:24.360034 (+ 271us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.237.62:40387: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40865/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":362,"threads_started":1}
I20260620 22:59:24.362003 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 8261
I20260620 22:59:24.363294 8322 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:24.456156 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 8261
2026-06-20T22:59:24Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (749 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260620 22:59:24.511718 2996 mini_oidc.cc:152] Starting JWKS server
I20260620 22:59:24.513655 2996 webserver.cc:460] Webserver started at https://127.0.0.1:46827/ using document root <none> and password file <none>
I20260620 22:59:24.513936 2996 mini_oidc.cc:189] Starting OIDC Discovery server
I20260620 22:59:24.514670 2996 webserver.cc:460] Webserver started at http://127.0.0.1:42149/ using document root <none> and password file <none>
2026-06-20T22:59:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T22:59:24Z Disabled control of system clock
I20260620 22:59:24.541759 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:43651
--webserver_interface=127.2.237.62
--webserver_port=0
--builtin_ntp_servers=127.2.237.20:39585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:43651
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46827/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260620 22:59:24.866362 8365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:24.866755 8365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:24.866945 8365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:24.874389 8365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 22:59:24.874603 8365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:24.874701 8365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:24.874806 8365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 22:59:24.874897 8365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 22:59:24.874996 8365 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 22:59:24.875072 8365 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46827/jwks.json
I20260620 22:59:24.884544 8365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:39585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:43651
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:43651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46827/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:24.885946 8365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:24.887879 8365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:24.900130 8370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:24.900104 8371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:24.900070 8373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:24.900975 8365 server_base.cc:1034] running on GCE node
I20260620 22:59:24.901697 8365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:24.903100 8365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:24.904299 8365 hybrid_clock.cc:648] HybridClock initialized: now 1781996364904262 us; error 28 us; skew 500 ppm
I20260620 22:59:24.904596 8365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 22:59:24.906837 8365 webserver.cc:460] Webserver started at http://127.2.237.62:46625/ using document root <none> and password file <none>
I20260620 22:59:24.907495 8365 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:24.907577 8365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:24.907841 8365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:24.910683 8365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "c4df6b2129a745d4b51dc5d049222e92"
format_stamp: "Formatted at 2026-06-20 22:59:24 on dist-test-slave-f63b"
server_key: "aca2009d33846b7451a57961cf942fba"
server_key_iv: "12a38180026a322058787db11b8e7b78"
server_key_version: "encryptionkey@0"
I20260620 22:59:24.911408 8365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "c4df6b2129a745d4b51dc5d049222e92"
format_stamp: "Formatted at 2026-06-20 22:59:24 on dist-test-slave-f63b"
server_key: "aca2009d33846b7451a57961cf942fba"
server_key_iv: "12a38180026a322058787db11b8e7b78"
server_key_version: "encryptionkey@0"
I20260620 22:59:24.916237 8365 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260620 22:59:24.919414 8379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:24.920660 8365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260620 22:59:24.920790 8365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "c4df6b2129a745d4b51dc5d049222e92"
format_stamp: "Formatted at 2026-06-20 22:59:24 on dist-test-slave-f63b"
server_key: "aca2009d33846b7451a57961cf942fba"
server_key_iv: "12a38180026a322058787db11b8e7b78"
server_key_version: "encryptionkey@0"
I20260620 22:59:24.920953 8365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:24.933218 8365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:24.933961 8365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:24.934211 8365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:24.951973 8365 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:43651
I20260620 22:59:24.951982 8430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:43651 every 8 connection(s)
I20260620 22:59:24.953339 8365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 22:59:24.957095 8431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:24.959259 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 8365
I20260620 22:59:24.959478 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 22:59:24.960254 2996 external_mini_cluster.cc:1428] Setting key 86882ab719ae415e7b8f534be5be0590
I20260620 22:59:24.969105 8431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4df6b2129a745d4b51dc5d049222e92: Bootstrap starting.
I20260620 22:59:24.972259 8431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4df6b2129a745d4b51dc5d049222e92: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:24.973341 8431 log.cc:824] T 00000000000000000000000000000000 P c4df6b2129a745d4b51dc5d049222e92: Log is configured to *not* fsync() on all Append() calls
I20260620 22:59:24.973675 8434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:24.962209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37198 (local address 127.2.237.62:43651)
0620 22:59:24.963272 (+ 1063us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:24.963286 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:24.963330 (+ 44us) server_negotiation.cc:407] Connection header received
0620 22:59:24.964951 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:24.964973 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:24.965424 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:24.965819 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:24.966409 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:24.967782 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:24.969067 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:24.969813 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:24.970013 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:24.971297 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:24.971305 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:24.971318 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 22:59:24.971882 (+ 564us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:24.971968 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 22:59:24.971989 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:24.972128 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:24.972350 (+ 222us) server_negotiation.cc:299] Negotiation successful
0620 22:59:24.972719 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":389,"threads_started":1}
I20260620 22:59:24.976380 8431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4df6b2129a745d4b51dc5d049222e92: No bootstrap required, opened a new log
I20260620 22:59:24.977336 2996 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 22:59:24.984175 8431 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4df6b2129a745d4b51dc5d049222e92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4df6b2129a745d4b51dc5d049222e92" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43651 } }
I20260620 22:59:24.984570 8431 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4df6b2129a745d4b51dc5d049222e92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:24.984675 8431 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4df6b2129a745d4b51dc5d049222e92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4df6b2129a745d4b51dc5d049222e92, State: Initialized, Role: FOLLOWER
I20260620 22:59:24.985219 8431 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4df6b2129a745d4b51dc5d049222e92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4df6b2129a745d4b51dc5d049222e92" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43651 } }
I20260620 22:59:24.985466 8431 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4df6b2129a745d4b51dc5d049222e92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 22:59:24.985611 8431 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4df6b2129a745d4b51dc5d049222e92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 22:59:24.985774 8431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4df6b2129a745d4b51dc5d049222e92 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:24.988071 8431 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4df6b2129a745d4b51dc5d049222e92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4df6b2129a745d4b51dc5d049222e92" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43651 } }
I20260620 22:59:24.988641 8431 leader_election.cc:302] T 00000000000000000000000000000000 P c4df6b2129a745d4b51dc5d049222e92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4df6b2129a745d4b51dc5d049222e92; no voters:
I20260620 22:59:24.989240 8431 leader_election.cc:288] T 00000000000000000000000000000000 P c4df6b2129a745d4b51dc5d049222e92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 22:59:24.989344 8436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4df6b2129a745d4b51dc5d049222e92 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:59:24.990156 8436 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4df6b2129a745d4b51dc5d049222e92 [term 1 LEADER]: Becoming Leader. State: Replica: c4df6b2129a745d4b51dc5d049222e92, State: Running, Role: LEADER
I20260620 22:59:24.990736 8436 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4df6b2129a745d4b51dc5d049222e92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4df6b2129a745d4b51dc5d049222e92" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43651 } }
I20260620 22:59:24.991366 8431 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4df6b2129a745d4b51dc5d049222e92 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 22:59:24.993825 8437 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4df6b2129a745d4b51dc5d049222e92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4df6b2129a745d4b51dc5d049222e92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4df6b2129a745d4b51dc5d049222e92" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43651 } } }
I20260620 22:59:24.994433 8437 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4df6b2129a745d4b51dc5d049222e92 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:24.993973 8438 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4df6b2129a745d4b51dc5d049222e92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4df6b2129a745d4b51dc5d049222e92. Latest consensus state: current_term: 1 leader_uuid: "c4df6b2129a745d4b51dc5d049222e92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4df6b2129a745d4b51dc5d049222e92" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43651 } } }
I20260620 22:59:24.995234 8438 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4df6b2129a745d4b51dc5d049222e92 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:24.996776 8443 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 22:59:25.001430 8443 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 22:59:25.010352 8443 catalog_manager.cc:1347] Generated new cluster ID: c03bd27e0ae243cca4876d7788a9813b
I20260620 22:59:25.010478 8443 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 22:59:25.027933 8443 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 22:59:25.029369 8443 catalog_manager.cc:1504] Loading token signing keys...
I20260620 22:59:25.047372 8443 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4df6b2129a745d4b51dc5d049222e92: Generated new TSK 0
I20260620 22:59:25.048271 8443 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 22:59:25.071867 8434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:25.063336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37200 (local address 127.2.237.62:43651)
0620 22:59:25.063580 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:25.063586 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:25.063950 (+ 364us) server_negotiation.cc:407] Connection header received
0620 22:59:25.064219 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:25.064223 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:25.064298 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:25.064432 (+ 134us) server_negotiation.cc:226] Negotiated authn=JWT
0620 22:59:25.064949 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:25.065646 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:25.067733 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:25.068181 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:25.068275 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:25.068872 (+ 597us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 22:59:25.071260 (+ 2388us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46827/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0620 22:59:25.071578 (+ 318us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37200: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46827/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":84}
W20260620 22:59:25.071982 8434 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37200: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46827/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260620 22:59:25.074043 8460 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 22:59:25.063135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43651 (local address 127.0.0.1:37200)
0620 22:59:25.063822 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:25.063845 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:25.064026 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:25.064480 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:25.064485 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:25.064499 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0620 22:59:25.064796 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:25.064807 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:25.065784 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:25.065790 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:25.067590 (+ 1800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:25.067606 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:25.068388 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:25.068395 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:25.068593 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:25.068642 (+ 49us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 22:59:25.071484 (+ 2842us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46827/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0620 22:59:25.073568 (+ 2084us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.237.62:43651: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46827/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":532,"mutex_wait_us":1785,"thread_start_us":375,"threads_started":1}
I20260620 22:59:25.075706 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 8365
I20260620 22:59:25.076907 8426 generic_service.cc:196] Checking for leaks (request via RPC)
E20260620 22:59:25.087385 8461 webserver.cc:533] Webserver: sslize failed:
E20260620 22:59:25.087541 8461 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260620 22:59:25.170285 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 8365
2026-06-20T22:59:25Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (714 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-20T22:59:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T22:59:25Z Disabled control of system clock
I20260620 22:59:25.253124 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:35385
--webserver_interface=127.2.237.62
--webserver_port=0
--builtin_ntp_servers=127.2.237.20:38055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:35385
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996318807807-2996-0/testchainca.cert with env {}
W20260620 22:59:25.573822 8466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:25.574186 8466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:25.574354 8466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:25.581701 8466 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 22:59:25.581851 8466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:25.581945 8466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:25.582031 8466 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 22:59:25.582121 8466 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260620 22:59:25.582211 8466 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260620 22:59:25.582284 8466 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260620 22:59:25.592154 8466 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:38055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:35385
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:35385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996318807807-2996-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:25.593551 8466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:25.594933 8466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:25.607117 8471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:25.607296 8474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:25.607309 8472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:25.608425 8466 server_base.cc:1034] running on GCE node
I20260620 22:59:25.609256 8466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:25.610749 8466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:25.611979 8466 hybrid_clock.cc:648] HybridClock initialized: now 1781996365611933 us; error 41 us; skew 500 ppm
I20260620 22:59:25.612326 8466 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 22:59:25.614642 8466 webserver.cc:460] Webserver started at http://127.2.237.62:34181/ using document root <none> and password file <none>
I20260620 22:59:25.615334 8466 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:25.615466 8466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:25.615780 8466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:25.618551 8466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "4ac8666c737946bc86029d4de316bb51"
format_stamp: "Formatted at 2026-06-20 22:59:25 on dist-test-slave-f63b"
server_key: "22f829d3dd2fa7c5577b676595c7a986"
server_key_iv: "53fb5281ecdbcc160fe90b13db3bde51"
server_key_version: "encryptionkey@0"
I20260620 22:59:25.619405 8466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "4ac8666c737946bc86029d4de316bb51"
format_stamp: "Formatted at 2026-06-20 22:59:25 on dist-test-slave-f63b"
server_key: "22f829d3dd2fa7c5577b676595c7a986"
server_key_iv: "53fb5281ecdbcc160fe90b13db3bde51"
server_key_version: "encryptionkey@0"
I20260620 22:59:25.624399 8466 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260620 22:59:25.627797 8480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:25.629225 8466 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 22:59:25.629443 8466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "4ac8666c737946bc86029d4de316bb51"
format_stamp: "Formatted at 2026-06-20 22:59:25 on dist-test-slave-f63b"
server_key: "22f829d3dd2fa7c5577b676595c7a986"
server_key_iv: "53fb5281ecdbcc160fe90b13db3bde51"
server_key_version: "encryptionkey@0"
I20260620 22:59:25.629699 8466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:25.668125 8466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:25.669093 8466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:25.669422 8466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:25.688952 8466 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:35385
I20260620 22:59:25.688972 8531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:35385 every 8 connection(s)
I20260620 22:59:25.690456 8466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 22:59:25.691831 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 8466
I20260620 22:59:25.692040 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 22:59:25.692768 2996 external_mini_cluster.cc:1428] Setting key 08d203f9f7058def7d514d4fbfed83ac
I20260620 22:59:25.695143 8532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:25.708197 8535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:25.694622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48604 (local address 127.2.237.62:35385)
0620 22:59:25.695757 (+ 1135us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:25.695771 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:25.695821 (+ 50us) server_negotiation.cc:407] Connection header received
0620 22:59:25.698008 (+ 2187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:25.698036 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:25.698673 (+ 637us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:25.699110 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:25.699640 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:25.701681 (+ 2041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:25.703051 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:25.704037 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:25.704216 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:25.705349 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:25.705359 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:25.705377 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 22:59:25.706100 (+ 723us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:25.706187 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 22:59:25.706218 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:25.706353 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:25.706566 (+ 213us) server_negotiation.cc:299] Negotiation successful
0620 22:59:25.706926 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":378,"threads_started":1}
I20260620 22:59:25.709220 8532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ac8666c737946bc86029d4de316bb51: Bootstrap starting.
I20260620 22:59:25.711935 2996 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 22:59:25.712502 8532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ac8666c737946bc86029d4de316bb51: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:25.713469 8532 log.cc:824] T 00000000000000000000000000000000 P 4ac8666c737946bc86029d4de316bb51: Log is configured to *not* fsync() on all Append() calls
I20260620 22:59:25.716476 8532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ac8666c737946bc86029d4de316bb51: No bootstrap required, opened a new log
I20260620 22:59:25.724211 8532 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ac8666c737946bc86029d4de316bb51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ac8666c737946bc86029d4de316bb51" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 35385 } }
I20260620 22:59:25.724560 8532 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ac8666c737946bc86029d4de316bb51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:25.724709 8532 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ac8666c737946bc86029d4de316bb51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ac8666c737946bc86029d4de316bb51, State: Initialized, Role: FOLLOWER
I20260620 22:59:25.725282 8532 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ac8666c737946bc86029d4de316bb51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ac8666c737946bc86029d4de316bb51" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 35385 } }
I20260620 22:59:25.725510 8532 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ac8666c737946bc86029d4de316bb51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 22:59:25.725656 8532 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ac8666c737946bc86029d4de316bb51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 22:59:25.725836 8532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ac8666c737946bc86029d4de316bb51 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:25.728263 8532 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ac8666c737946bc86029d4de316bb51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ac8666c737946bc86029d4de316bb51" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 35385 } }
I20260620 22:59:25.728811 8532 leader_election.cc:302] T 00000000000000000000000000000000 P 4ac8666c737946bc86029d4de316bb51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ac8666c737946bc86029d4de316bb51; no voters:
I20260620 22:59:25.729408 8532 leader_election.cc:288] T 00000000000000000000000000000000 P 4ac8666c737946bc86029d4de316bb51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 22:59:25.729611 8537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ac8666c737946bc86029d4de316bb51 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:59:25.730384 8537 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ac8666c737946bc86029d4de316bb51 [term 1 LEADER]: Becoming Leader. State: Replica: 4ac8666c737946bc86029d4de316bb51, State: Running, Role: LEADER
I20260620 22:59:25.730966 8537 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ac8666c737946bc86029d4de316bb51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ac8666c737946bc86029d4de316bb51" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 35385 } }
I20260620 22:59:25.731455 8532 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ac8666c737946bc86029d4de316bb51 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 22:59:25.734354 8539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ac8666c737946bc86029d4de316bb51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ac8666c737946bc86029d4de316bb51. Latest consensus state: current_term: 1 leader_uuid: "4ac8666c737946bc86029d4de316bb51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ac8666c737946bc86029d4de316bb51" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 35385 } } }
I20260620 22:59:25.734232 8538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ac8666c737946bc86029d4de316bb51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ac8666c737946bc86029d4de316bb51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ac8666c737946bc86029d4de316bb51" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 35385 } } }
I20260620 22:59:25.734823 8538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ac8666c737946bc86029d4de316bb51 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:25.734823 8539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ac8666c737946bc86029d4de316bb51 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:25.735728 8546 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 22:59:25.742331 8546 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 22:59:25.751809 8546 catalog_manager.cc:1347] Generated new cluster ID: c914534b04d0422a993be222e01848f3
I20260620 22:59:25.751942 8546 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 22:59:25.759845 8546 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 22:59:25.761564 8546 catalog_manager.cc:1504] Loading token signing keys...
I20260620 22:59:25.773376 8546 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ac8666c737946bc86029d4de316bb51: Generated new TSK 0
I20260620 22:59:25.774425 8546 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 22:59:26.667622 8535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:25.791965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48608 (local address 127.2.237.62:35385)
0620 22:59:25.792164 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:25.792172 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:25.792574 (+ 402us) server_negotiation.cc:407] Connection header received
0620 22:59:25.792765 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:25.792772 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:25.792866 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:25.793051 (+ 185us) server_negotiation.cc:226] Negotiated authn=JWT
0620 22:59:25.793462 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:25.794661 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:25.796005 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:25.796707 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:25.796845 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:25.797318 (+ 473us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0620 22:59:26.667039 (+869721us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0620 22:59:26.667393 (+ 354us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48608: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":67}
W20260620 22:59:26.667727 8535 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48608: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260620 22:59:26.667927 8561 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 22:59:25.791822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:35385 (local address 127.0.0.1:48608)
0620 22:59:25.792467 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:25.792483 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:25.792602 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:25.793050 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:25.793053 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:25.793062 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0620 22:59:25.793315 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:25.793322 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:25.794798 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:25.794802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:25.795861 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:25.795872 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:25.796873 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:25.796877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:25.797010 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:25.797041 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0620 22:59:26.667334 (+870293us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0620 22:59:26.667565 (+ 231us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.237.62:35385: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":344,"threads_started":1}
I20260620 22:59:26.668992 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 8466
I20260620 22:59:26.670624 8527 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:26.778170 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 8466
2026-06-20T22:59:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1588 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (405 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (407 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 22:59:27 dist-test-slave-f63b krb5kdc[8576](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 22:59:27 dist-test-slave-f63b krb5kdc[8576](info): set up 2 sockets
Jun 20 22:59:27 dist-test-slave-f63b krb5kdc[8576](info): commencing operation
krb5kdc: starting...
W20260620 22:59:28.679275 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 22:59:28 dist-test-slave-f63b krb5kdc[8576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T22:59:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T22:59:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
I20260620 22:59:28.886482 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:40601
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:37327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:40601
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:29.207836 8589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:29.208132 8589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:29.208317 8589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:29.215569 8589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 22:59:29.215682 8589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:29.215768 8589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:29.215874 8589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 22:59:29.215956 8589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 22:59:29.225733 8589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:37327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:40601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:40601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:29.227092 8589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:29.228395 8589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:29.240166 8595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:29.240309 8594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:29.240278 8597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:29.241168 8589 server_base.cc:1034] running on GCE node
I20260620 22:59:29.242015 8589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:29.243592 8589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:29.244812 8589 hybrid_clock.cc:648] HybridClock initialized: now 1781996369244761 us; error 47 us; skew 500 ppm
I20260620 22:59:29.245193 8589 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:29 dist-test-slave-f63b krb5kdc[8576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996369, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:29.251215 8589 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 22:59:29.252818 8589 webserver.cc:460] Webserver started at http://127.2.237.62:42351/ using document root <none> and password file <none>
I20260620 22:59:29.253787 8589 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:29.253924 8589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:29.254277 8589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:29.258724 8589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "d2516f780dc3448c9a824b4613d51789"
format_stamp: "Formatted at 2026-06-20 22:59:29 on dist-test-slave-f63b"
server_key: "f375e9635945bd8ddaf7573727cc1625"
server_key_iv: "52b1bb63122f09762cb8a6bcda16da50"
server_key_version: "encryptionkey@0"
I20260620 22:59:29.259706 8589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "d2516f780dc3448c9a824b4613d51789"
format_stamp: "Formatted at 2026-06-20 22:59:29 on dist-test-slave-f63b"
server_key: "f375e9635945bd8ddaf7573727cc1625"
server_key_iv: "52b1bb63122f09762cb8a6bcda16da50"
server_key_version: "encryptionkey@0"
I20260620 22:59:29.265092 8589 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260620 22:59:29.268476 8604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:29.269714 8589 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 22:59:29.269883 8589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "d2516f780dc3448c9a824b4613d51789"
format_stamp: "Formatted at 2026-06-20 22:59:29 on dist-test-slave-f63b"
server_key: "f375e9635945bd8ddaf7573727cc1625"
server_key_iv: "52b1bb63122f09762cb8a6bcda16da50"
server_key_version: "encryptionkey@0"
I20260620 22:59:29.270073 8589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:29.295171 8589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:29.299335 8589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:29.299628 8589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:29.318383 8589 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:40601
I20260620 22:59:29.318416 8655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:40601 every 8 connection(s)
I20260620 22:59:29.319832 8589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 22:59:29.323658 8656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:29.327113 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 8589
I20260620 22:59:29.327311 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 22:59:29.328138 2996 external_mini_cluster.cc:1428] Setting key d95fc349736f97a7f0dd7d1d0de63c0f
I20260620 22:59:29.336807 8656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2516f780dc3448c9a824b4613d51789: Bootstrap starting.
I20260620 22:59:29.340674 8656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2516f780dc3448c9a824b4613d51789: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:29.342012 8656 log.cc:824] T 00000000000000000000000000000000 P d2516f780dc3448c9a824b4613d51789: Log is configured to *not* fsync() on all Append() calls
Jun 20 22:59:29 dist-test-slave-f63b krb5kdc[8576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:59:29.345175 8656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2516f780dc3448c9a824b4613d51789: No bootstrap required, opened a new log
I20260620 22:59:29.352603 8659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:29.330274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56046 (local address 127.2.237.62:40601)
0620 22:59:29.331073 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:29.331084 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:29.331130 (+ 46us) server_negotiation.cc:407] Connection header received
0620 22:59:29.332708 (+ 1578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:29.332731 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:29.333174 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:29.333508 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:29.335202 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:29.336674 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:29.338441 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:29.339281 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:29.339434 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:29.344518 (+ 5084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:29.344526 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:29.344542 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:29.344593 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:29.348085 (+ 3492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:29.348898 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:29.348905 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:29.348912 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:29.349116 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:29.349560 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:29.349577 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:29.349580 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:29.350160 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:29.350518 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:29.351025 (+ 507us) server_negotiation.cc:299] Negotiation successful
0620 22:59:29.351495 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":410,"threads_started":1}
I20260620 22:59:29.353099 8656 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2516f780dc3448c9a824b4613d51789 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2516f780dc3448c9a824b4613d51789" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 40601 } }
I20260620 22:59:29.353495 8656 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2516f780dc3448c9a824b4613d51789 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:29.353607 8656 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2516f780dc3448c9a824b4613d51789 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2516f780dc3448c9a824b4613d51789, State: Initialized, Role: FOLLOWER
I20260620 22:59:29.354275 8656 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2516f780dc3448c9a824b4613d51789 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2516f780dc3448c9a824b4613d51789" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 40601 } }
I20260620 22:59:29.354516 8656 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2516f780dc3448c9a824b4613d51789 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 22:59:29.354687 8656 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2516f780dc3448c9a824b4613d51789 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 22:59:29.354858 8656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2516f780dc3448c9a824b4613d51789 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:29.357280 8656 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2516f780dc3448c9a824b4613d51789 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2516f780dc3448c9a824b4613d51789" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 40601 } }
I20260620 22:59:29.357759 8656 leader_election.cc:302] T 00000000000000000000000000000000 P d2516f780dc3448c9a824b4613d51789 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2516f780dc3448c9a824b4613d51789; no voters:
I20260620 22:59:29.358361 8656 leader_election.cc:288] T 00000000000000000000000000000000 P d2516f780dc3448c9a824b4613d51789 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 22:59:29.358493 8661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2516f780dc3448c9a824b4613d51789 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:59:29.359669 8661 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2516f780dc3448c9a824b4613d51789 [term 1 LEADER]: Becoming Leader. State: Replica: d2516f780dc3448c9a824b4613d51789, State: Running, Role: LEADER
I20260620 22:59:29.360216 8661 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2516f780dc3448c9a824b4613d51789 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2516f780dc3448c9a824b4613d51789" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 40601 } }
I20260620 22:59:29.360426 8656 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2516f780dc3448c9a824b4613d51789 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 22:59:29.363451 8663 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2516f780dc3448c9a824b4613d51789 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2516f780dc3448c9a824b4613d51789. Latest consensus state: current_term: 1 leader_uuid: "d2516f780dc3448c9a824b4613d51789" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2516f780dc3448c9a824b4613d51789" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 40601 } } }
I20260620 22:59:29.363927 8663 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2516f780dc3448c9a824b4613d51789 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:29.364452 8662 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2516f780dc3448c9a824b4613d51789 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2516f780dc3448c9a824b4613d51789" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2516f780dc3448c9a824b4613d51789" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 40601 } } }
I20260620 22:59:29.364805 8662 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2516f780dc3448c9a824b4613d51789 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:29.367543 8669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 22:59:29.372596 8669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 22:59:29.381692 8669 catalog_manager.cc:1347] Generated new cluster ID: 4206a6880ba646c4b30950c771d806fb
I20260620 22:59:29.381819 8669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 22:59:29.401963 8669 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 22:59:29.403986 8669 catalog_manager.cc:1504] Loading token signing keys...
I20260620 22:59:29.419014 8669 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2516f780dc3448c9a824b4613d51789: Generated new TSK 0
I20260620 22:59:29.420141 8669 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.237.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
I20260620 22:59:29.478552 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:0
--local_ip_for_outbound_sockets=127.2.237.1
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:40601
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:37327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:29.821570 8682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:29.821897 8682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:29.822062 8682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:29.829535 8682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:29.829694 8682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:29.829881 8682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 22:59:29.839969 8682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:37327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:40601
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:29.841571 8682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:29.842905 8682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:29.856338 8688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:29.856268 8690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:29.856370 8687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:29.857023 8682 server_base.cc:1034] running on GCE node
I20260620 22:59:29.857692 8682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:29.858618 8682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:29.859886 8682 hybrid_clock.cc:648] HybridClock initialized: now 1781996369859807 us; error 61 us; skew 500 ppm
I20260620 22:59:29.860252 8682 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:29 dist-test-slave-f63b krb5kdc[8576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996369, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:29.863507 8682 init.cc:377] Logged in from keytab as kudu/127.2.237.1@KRBTEST.COM (short username kudu)
I20260620 22:59:29.864982 8682 webserver.cc:460] Webserver started at http://127.2.237.1:40499/ using document root <none> and password file <none>
I20260620 22:59:29.865677 8682 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:29.865816 8682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:29.866117 8682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:29.869314 8682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-0/data/instance:
uuid: "0432642838e943938a68afb8c08a7866"
format_stamp: "Formatted at 2026-06-20 22:59:29 on dist-test-slave-f63b"
server_key: "fbf26cae44d459c3324efef42c17edb9"
server_key_iv: "2d78256f603931f883209072b0710194"
server_key_version: "encryptionkey@0"
I20260620 22:59:29.870182 8682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance:
uuid: "0432642838e943938a68afb8c08a7866"
format_stamp: "Formatted at 2026-06-20 22:59:29 on dist-test-slave-f63b"
server_key: "fbf26cae44d459c3324efef42c17edb9"
server_key_iv: "2d78256f603931f883209072b0710194"
server_key_version: "encryptionkey@0"
I20260620 22:59:29.875340 8682 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260620 22:59:29.878871 8697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:29.880249 8682 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260620 22:59:29.880412 8682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "0432642838e943938a68afb8c08a7866"
format_stamp: "Formatted at 2026-06-20 22:59:29 on dist-test-slave-f63b"
server_key: "fbf26cae44d459c3324efef42c17edb9"
server_key_iv: "2d78256f603931f883209072b0710194"
server_key_version: "encryptionkey@0"
I20260620 22:59:29.880605 8682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:29.894259 8682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:29.898633 8682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:29.898921 8682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:29.899879 8682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:59:29.901162 8682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:29.901233 8682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:29.901369 8682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:29.901436 8682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:29.934680 8682 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:38961
I20260620 22:59:29.934741 8809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:38961 every 8 connection(s)
I20260620 22:59:29.936051 8682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 22:59:29.939623 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 8682
I20260620 22:59:29.939846 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance
I20260620 22:59:29.940542 2996 external_mini_cluster.cc:1428] Setting key d1d846846efe73e91864d4de063dc793
Jun 20 22:59:29 dist-test-slave-f63b krb5kdc[8576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996369, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:59:29.955880 8812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:29.938554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:58597 (local address 127.2.237.62:40601)
0620 22:59:29.939044 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:29.939048 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:29.940110 (+ 1062us) server_negotiation.cc:407] Connection header received
0620 22:59:29.942006 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:29.942012 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:29.942083 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:29.942261 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:29.943823 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:29.944550 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:29.946101 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:29.946512 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:29.946701 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:29.950329 (+ 3628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:29.950334 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:29.950338 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:29.950370 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:29.952881 (+ 2511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:29.953611 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:29.953616 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:29.953619 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:29.953749 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:29.954159 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:29.954166 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:29.954169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:29.954421 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:29.954566 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:29.955310 (+ 744us) server_negotiation.cc:299] Negotiation successful
0620 22:59:29.955524 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":241,"threads_started":1}
I20260620 22:59:29.956934 8813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:29.939069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:40601 (local address 127.2.237.1:58597)
0620 22:59:29.939933 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:29.939984 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:29.941754 (+ 1770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:29.942473 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:29.942481 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:29.943055 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:29.943651 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:29.943669 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:29.944726 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:29.944730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:29.945947 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:29.945958 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:29.946787 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:29.946796 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:29.947034 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:29.947726 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:29.947749 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:29.950114 (+ 2365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:29.953031 (+ 2917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:29.953040 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:29.953057 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:29.953486 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:29.953857 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:29.953863 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:29.953867 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:29.954051 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:29.954557 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:29.954565 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:29.954939 (+ 374us) client_negotiation.cc:769] Sending connection context
0620 22:59:29.955204 (+ 265us) client_negotiation.cc:240] Negotiation successful
0620 22:59:29.955753 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":356,"threads_started":1}
I20260620 22:59:29.958680 8810 heartbeater.cc:347] Connected to a master server at 127.2.237.62:40601
WARNING: no policy specified for kudu/127.2.237.2@KRBTEST.COM; defaulting to no policy
I20260620 22:59:29.958981 8810 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:29.959728 8810 heartbeater.cc:511] Master 127.2.237.62:40601 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.2@KRBTEST.COM" created.
I20260620 22:59:29.962206 8621 ts_manager.cc:194] Registered new tserver with Master: 0432642838e943938a68afb8c08a7866 (127.2.237.1:38961)
I20260620 22:59:29.964391 8621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.1@KRBTEST.COM'} at 127.2.237.1:58597
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
I20260620 22:59:29.987280 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:0
--local_ip_for_outbound_sockets=127.2.237.2
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:40601
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:37327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:30.308151 8816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:30.308471 8816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:30.308566 8816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:30.317036 8816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:30.317162 8816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:30.317281 8816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 22:59:30.331826 8816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:37327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:40601
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:30.333148 8816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:30.334465 8816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:30.347482 8824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:30.347533 8821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:30.347535 8822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:30.353964 8816 server_base.cc:1034] running on GCE node
I20260620 22:59:30.354749 8816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:30.355754 8816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:30.356982 8816 hybrid_clock.cc:648] HybridClock initialized: now 1781996370356939 us; error 41 us; skew 500 ppm
I20260620 22:59:30.357309 8816 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:30 dist-test-slave-f63b krb5kdc[8576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996370, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:30.360450 8816 init.cc:377] Logged in from keytab as kudu/127.2.237.2@KRBTEST.COM (short username kudu)
I20260620 22:59:30.362063 8816 webserver.cc:460] Webserver started at http://127.2.237.2:37311/ using document root <none> and password file <none>
I20260620 22:59:30.362862 8816 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:30.362999 8816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:30.363371 8816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:30.366240 8816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-1/data/instance:
uuid: "4a25913b30134692be43f4023714296b"
format_stamp: "Formatted at 2026-06-20 22:59:30 on dist-test-slave-f63b"
server_key: "0b537cb75182fc2f8a43ada583c45db9"
server_key_iv: "82a8d638fbb2ebc3eb4dc1e0f764e2c3"
server_key_version: "encryptionkey@0"
I20260620 22:59:30.367044 8816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance:
uuid: "4a25913b30134692be43f4023714296b"
format_stamp: "Formatted at 2026-06-20 22:59:30 on dist-test-slave-f63b"
server_key: "0b537cb75182fc2f8a43ada583c45db9"
server_key_iv: "82a8d638fbb2ebc3eb4dc1e0f764e2c3"
server_key_version: "encryptionkey@0"
I20260620 22:59:30.372252 8816 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260620 22:59:30.375810 8831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:30.377130 8816 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260620 22:59:30.377310 8816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "4a25913b30134692be43f4023714296b"
format_stamp: "Formatted at 2026-06-20 22:59:30 on dist-test-slave-f63b"
server_key: "0b537cb75182fc2f8a43ada583c45db9"
server_key_iv: "82a8d638fbb2ebc3eb4dc1e0f764e2c3"
server_key_version: "encryptionkey@0"
I20260620 22:59:30.377514 8816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:30.396226 8816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:30.400154 8816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:30.400444 8816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:30.401371 8816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:59:30.402711 8816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:30.402812 8816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:30.402958 8816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:30.403028 8816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:30.436193 8816 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:36601
I20260620 22:59:30.436293 8943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:36601 every 8 connection(s)
I20260620 22:59:30.437589 8816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 22:59:30.447378 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 8816
I20260620 22:59:30.447553 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance
I20260620 22:59:30.447996 2996 external_mini_cluster.cc:1428] Setting key 2179569d7ba8d605a069878fa9ee7793
Jun 20 22:59:30 dist-test-slave-f63b krb5kdc[8576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996370, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:59:30.458014 8812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:30.440348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:48601 (local address 127.2.237.62:40601)
0620 22:59:30.440601 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:30.440605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:30.441784 (+ 1179us) server_negotiation.cc:407] Connection header received
0620 22:59:30.443686 (+ 1902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:30.443692 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:30.443786 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:30.443901 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:30.445502 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:30.446215 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:30.448011 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:30.448529 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:30.448706 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:30.452262 (+ 3556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:30.452267 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:30.452271 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:30.452304 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:30.454918 (+ 2614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:30.455661 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:30.455666 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:30.455676 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:30.455795 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:30.456228 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:30.456233 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:30.456236 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:30.456504 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:30.456684 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:30.457414 (+ 730us) server_negotiation.cc:299] Negotiation successful
0620 22:59:30.457651 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260620 22:59:30.459095 8946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:30.440849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:40601 (local address 127.2.237.2:48601)
0620 22:59:30.441602 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:30.441649 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:30.443392 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:30.444150 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:30.444158 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:30.444677 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:30.445300 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:30.445326 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:30.446368 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:30.446372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:30.447862 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:30.447874 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:30.448816 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:30.448823 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:30.449051 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:30.449749 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:30.449777 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:30.452014 (+ 2237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:30.455061 (+ 3047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:30.455072 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:30.455091 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:30.455517 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:30.455906 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:30.455912 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:30.455916 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:30.456108 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:30.456647 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:30.456655 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:30.457013 (+ 358us) client_negotiation.cc:769] Sending connection context
0620 22:59:30.457288 (+ 275us) client_negotiation.cc:240] Negotiation successful
0620 22:59:30.457841 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":293,"threads_started":1}
I20260620 22:59:30.460589 8944 heartbeater.cc:347] Connected to a master server at 127.2.237.62:40601
I20260620 22:59:30.460898 8944 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:30.461774 8944 heartbeater.cc:511] Master 127.2.237.62:40601 requested a full tablet report, sending...
I20260620 22:59:30.463317 8621 ts_manager.cc:194] Registered new tserver with Master: 4a25913b30134692be43f4023714296b (127.2.237.2:36601)
I20260620 22:59:30.464216 8621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.2@KRBTEST.COM'} at 127.2.237.2:48601
WARNING: no policy specified for kudu/127.2.237.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
I20260620 22:59:30.494082 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:0
--local_ip_for_outbound_sockets=127.2.237.3
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:40601
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:37327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:30.817472 8949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:30.817747 8949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:30.817905 8949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:30.825317 8949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:30.825435 8949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:30.825619 8949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 22:59:30.835510 8949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:37327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:40601
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:30.836838 8949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:30.838152 8949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:30.851104 8954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:30.851104 8955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:30.851186 8957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:30.851835 8949 server_base.cc:1034] running on GCE node
I20260620 22:59:30.852560 8949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:30.853506 8949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:30.854722 8949 hybrid_clock.cc:648] HybridClock initialized: now 1781996370854666 us; error 59 us; skew 500 ppm
I20260620 22:59:30.855036 8949 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:30 dist-test-slave-f63b krb5kdc[8576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996370, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:30.858016 8949 init.cc:377] Logged in from keytab as kudu/127.2.237.3@KRBTEST.COM (short username kudu)
I20260620 22:59:30.859488 8949 webserver.cc:460] Webserver started at http://127.2.237.3:34795/ using document root <none> and password file <none>
I20260620 22:59:30.860174 8949 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:30.860327 8949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:30.860641 8949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:30.863732 8949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-2/data/instance:
uuid: "0ad7972652294eeaaa5a58ac1c3d3b3f"
format_stamp: "Formatted at 2026-06-20 22:59:30 on dist-test-slave-f63b"
server_key: "146442fe82aafd138617792900ddd11c"
server_key_iv: "577ae0c280eb67a40245e95b5190d2bc"
server_key_version: "encryptionkey@0"
I20260620 22:59:30.864557 8949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance:
uuid: "0ad7972652294eeaaa5a58ac1c3d3b3f"
format_stamp: "Formatted at 2026-06-20 22:59:30 on dist-test-slave-f63b"
server_key: "146442fe82aafd138617792900ddd11c"
server_key_iv: "577ae0c280eb67a40245e95b5190d2bc"
server_key_version: "encryptionkey@0"
I20260620 22:59:30.869627 8949 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260620 22:59:30.873342 8964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:30.874804 8949 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260620 22:59:30.874965 8949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "0ad7972652294eeaaa5a58ac1c3d3b3f"
format_stamp: "Formatted at 2026-06-20 22:59:30 on dist-test-slave-f63b"
server_key: "146442fe82aafd138617792900ddd11c"
server_key_iv: "577ae0c280eb67a40245e95b5190d2bc"
server_key_version: "encryptionkey@0"
I20260620 22:59:30.875164 8949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:30.897096 8949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:30.901129 8949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:30.901423 8949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:30.902377 8949 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:59:30.903846 8949 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:30.903949 8949 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:30.904096 8949 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:30.904168 8949 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:30.937762 8949 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:42339
I20260620 22:59:30.937804 9076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:42339 every 8 connection(s)
I20260620 22:59:30.939107 8949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 22:59:30.944211 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 8949
I20260620 22:59:30.944413 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance
I20260620 22:59:30.944893 2996 external_mini_cluster.cc:1428] Setting key 3e4e68d4a880d739ac3d53032af7fb36
Jun 20 22:59:30 dist-test-slave-f63b krb5kdc[8576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996370, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:59:30.959860 8812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:30.941840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:49969 (local address 127.2.237.62:40601)
0620 22:59:30.942071 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:30.942076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:30.943448 (+ 1372us) server_negotiation.cc:407] Connection header received
0620 22:59:30.945466 (+ 2018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:30.945474 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:30.945582 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:30.945683 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:30.947499 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:30.948499 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:30.950050 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:30.950442 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:30.950540 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:30.954342 (+ 3802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:30.954346 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:30.954349 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:30.954382 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:30.956742 (+ 2360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:30.957413 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:30.957416 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:30.957418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:30.957531 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:30.957959 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:30.957962 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:30.957964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:30.958216 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:30.958324 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:30.959282 (+ 958us) server_negotiation.cc:299] Negotiation successful
0620 22:59:30.959506 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260620 22:59:30.961042 9079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:30.942287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:40601 (local address 127.2.237.3:49969)
0620 22:59:30.943246 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:30.943293 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:30.945186 (+ 1893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:30.945976 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:30.945985 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:30.946639 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:30.947327 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:30.947346 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:30.948688 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:30.948692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:30.949915 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:30.949927 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:30.950717 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:30.950724 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:30.950915 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:30.951537 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:30.951559 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:30.954148 (+ 2589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:30.956863 (+ 2715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:30.956870 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:30.956883 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:30.957300 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:30.957619 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:30.957622 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:30.957624 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:30.957820 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:30.958333 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:30.958339 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:30.958824 (+ 485us) client_negotiation.cc:769] Sending connection context
0620 22:59:30.959132 (+ 308us) client_negotiation.cc:240] Negotiation successful
0620 22:59:30.959657 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":499,"threads_started":1}
I20260620 22:59:30.962316 9077 heartbeater.cc:347] Connected to a master server at 127.2.237.62:40601
I20260620 22:59:30.962615 9077 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:30.963289 9077 heartbeater.cc:511] Master 127.2.237.62:40601 requested a full tablet report, sending...
I20260620 22:59:30.964603 8621 ts_manager.cc:194] Registered new tserver with Master: 0ad7972652294eeaaa5a58ac1c3d3b3f (127.2.237.3:42339)
I20260620 22:59:30.965371 8621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.3@KRBTEST.COM'} at 127.2.237.3:49969
I20260620 22:59:30.967633 8810 heartbeater.cc:503] Master 127.2.237.62:40601 was elected leader, sending a full tablet report...
I20260620 22:59:30.972769 2996 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/test-admin.keytab.
Jun 20 22:59:30 dist-test-slave-f63b krb5kdc[8576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:30.999675 2996 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260620 22:59:31.004765 9084 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 22:59:31.002243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58906 (local address 127.2.237.1:38961)
0620 22:59:31.002972 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:31.002980 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:31.003006 (+ 26us) server_negotiation.cc:407] Connection header received
0620 22:59:31.003064 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:31.003070 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:31.003306 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:31.003438 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:31.004298 (+ 860us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58906: BlockingRecv error: recv got EOF from 127.0.0.1:58906 (error 108)
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":440,"threads_started":1}
W20260620 22:59:31.004870 2996 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/krb5cc' of type FILE
Jun 20 22:59:31 dist-test-slave-f63b krb5kdc[8576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:31.006454 2996 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 20 22:59:31 dist-test-slave-f63b krb5kdc[8576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:31.007781 2996 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 20 22:59:31 dist-test-slave-f63b krb5kdc[8576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.1@KRBTEST.COM
I20260620 22:59:31.022419 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:31.009396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58910 (local address 127.2.237.1:38961)
0620 22:59:31.009571 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:31.009579 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:31.009736 (+ 157us) server_negotiation.cc:407] Connection header received
0620 22:59:31.009849 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:31.009852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:31.009908 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:31.009986 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:31.010999 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:31.011931 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:31.013111 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:31.013655 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:31.013759 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:31.017260 (+ 3501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:31.017272 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:31.017287 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:31.017326 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:31.019979 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:31.020592 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:31.020598 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:31.020600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:31.020713 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:31.021023 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:31.021026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:31.021028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:31.021353 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:31.021554 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:31.021883 (+ 329us) server_negotiation.cc:299] Negotiation successful
0620 22:59:31.022142 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260620 22:59:31.028120 9084 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 22:59:31.026358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58920 (local address 127.2.237.1:38961)
0620 22:59:31.026560 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:31.026565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:31.026799 (+ 234us) server_negotiation.cc:407] Connection header received
0620 22:59:31.026994 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:31.026999 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:31.027063 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:31.027162 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:31.027876 (+ 714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58920: BlockingRecv error: recv got EOF from 127.0.0.1:58920 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260620 22:59:31.028450 2996 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/krb5cc' of type FILE
Jun 20 22:59:31 dist-test-slave-f63b krb5kdc[8576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:31.029938 2996 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 20 22:59:31 dist-test-slave-f63b krb5kdc[8576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:31.031243 2996 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 20 22:59:31 dist-test-slave-f63b krb5kdc[8576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.1@KRBTEST.COM
I20260620 22:59:31.044824 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:31.032495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58924 (local address 127.2.237.1:38961)
0620 22:59:31.032700 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:31.032708 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:31.032963 (+ 255us) server_negotiation.cc:407] Connection header received
0620 22:59:31.033147 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:31.033156 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:31.033210 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:31.033309 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:31.034209 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:31.034985 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:31.036204 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:31.036626 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:31.036770 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:31.040085 (+ 3315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:31.040090 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:31.040093 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:31.040134 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:31.042376 (+ 2242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:31.043030 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:31.043036 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:31.043040 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:31.043163 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:31.043536 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:31.043543 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:31.043546 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:31.043810 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:31.044072 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:31.044348 (+ 276us) server_negotiation.cc:299] Negotiation successful
0620 22:59:31.044548 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260620 22:59:31.049273 9084 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 22:59:31.047502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58938 (local address 127.2.237.1:38961)
0620 22:59:31.047738 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:31.047743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:31.047970 (+ 227us) server_negotiation.cc:407] Connection header received
0620 22:59:31.048141 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:31.048147 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:31.048222 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:31.048336 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:31.049039 (+ 703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58938: BlockingRecv error: recv got EOF from 127.0.0.1:58938 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260620 22:59:31.049469 2996 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781996318807807-2996-0/krb5kdc/krb5cc' of type FILE
Jun 20 22:59:31 dist-test-slave-f63b krb5kdc[8576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:31.051314 2996 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 20 22:59:31 dist-test-slave-f63b krb5kdc[8576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:31.052812 2996 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 20 22:59:31 dist-test-slave-f63b krb5kdc[8576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.1@KRBTEST.COM
I20260620 22:59:31.067346 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:31.054092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58950 (local address 127.2.237.1:38961)
0620 22:59:31.054299 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:31.054305 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:31.054567 (+ 262us) server_negotiation.cc:407] Connection header received
0620 22:59:31.054802 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:31.054808 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:31.054880 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:31.054994 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:31.055926 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:31.056626 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:31.057818 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:31.058236 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:31.058360 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:31.061868 (+ 3508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:31.061873 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:31.061876 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:31.061921 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:31.064818 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:31.065432 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:31.065437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:31.065441 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:31.065596 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:31.065943 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:31.065948 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:31.065952 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:31.066207 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:31.066514 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:31.066818 (+ 304us) server_negotiation.cc:299] Negotiation successful
0620 22:59:31.067024 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260620 22:59:31.068915 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 8682
I20260620 22:59:31.080852 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:31.069639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58962 (local address 127.2.237.1:38961)
0620 22:59:31.069790 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:31.069794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:31.070266 (+ 472us) server_negotiation.cc:407] Connection header received
0620 22:59:31.070387 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:31.070391 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:31.070444 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:31.070517 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:31.071497 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:31.072145 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:31.073382 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:31.073804 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:31.073903 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:31.075865 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:31.075870 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:31.075873 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:31.075899 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:31.078329 (+ 2430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:31.079003 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:31.079010 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:31.079014 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:31.079140 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:31.079508 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:31.079513 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:31.079516 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:31.079764 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:31.080039 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:31.080325 (+ 286us) server_negotiation.cc:299] Negotiation successful
0620 22:59:31.080514 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 22:59:31.081422 8805 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:31.162444 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 8682
I20260620 22:59:31.178921 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 8816
Jun 20 22:59:31 dist-test-slave-f63b krb5kdc[8576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.2@KRBTEST.COM
I20260620 22:59:31.193807 9097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:31.179750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42864 (local address 127.2.237.2:36601)
0620 22:59:31.180341 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:31.180348 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:31.180374 (+ 26us) server_negotiation.cc:407] Connection header received
0620 22:59:31.180440 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:31.180447 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:31.180655 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:31.180801 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:31.181748 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:31.182988 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:31.184247 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:31.184794 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:31.184889 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:31.188590 (+ 3701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:31.188600 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:31.188616 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:31.188646 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:31.191385 (+ 2739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:31.191991 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:31.191998 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:31.192000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:31.192114 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:31.192534 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:31.192537 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:31.192539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:31.192856 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:31.193077 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:31.193329 (+ 252us) server_negotiation.cc:299] Negotiation successful
0620 22:59:31.193519 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":287,"threads_started":1}
I20260620 22:59:31.195084 8939 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:31.278048 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 8816
I20260620 22:59:31.293859 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 8949
Jun 20 22:59:31 dist-test-slave-f63b krb5kdc[8576](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.3@KRBTEST.COM
I20260620 22:59:31.309114 9099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:31.294836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44602 (local address 127.2.237.3:42339)
0620 22:59:31.295394 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:31.295401 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:31.295430 (+ 29us) server_negotiation.cc:407] Connection header received
0620 22:59:31.295493 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:31.295499 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:31.295710 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:31.295842 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:31.296805 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:31.297793 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:31.299084 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:31.299634 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:31.299732 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:31.303646 (+ 3914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:31.303657 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:31.303673 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:31.303705 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:31.306472 (+ 2767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:31.307187 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:31.307194 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:31.307196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:31.307326 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:31.307659 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:31.307662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:31.307664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:31.307984 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:31.308210 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:31.308537 (+ 327us) server_negotiation.cc:299] Negotiation successful
0620 22:59:31.308789 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":272,"threads_started":1}
I20260620 22:59:31.310477 9072 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:31.397177 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 8949
I20260620 22:59:31.412820 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 8589
I20260620 22:59:31.413970 8651 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:31.516309 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 8589
2026-06-20T22:59:31Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3929 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 22:59:31 dist-test-slave-f63b krb5kdc[9104](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 22:59:31 dist-test-slave-f63b krb5kdc[9104](info): set up 2 sockets
Jun 20 22:59:31 dist-test-slave-f63b krb5kdc[9104](info): commencing operation
krb5kdc: starting...
W20260620 22:59:32.615448 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.004s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 22:59:32 dist-test-slave-f63b krb5kdc[9104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T22:59:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T22:59:32Z Disabled control of system clock
WARNING: no policy specified for oryx/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.62.keytab.
Entry for principal oryx/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.62.keytab.
I20260620 22:59:32.830518 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:36895
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.62.keytab
--principal=oryx/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:39019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:36895
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:33.153446 9117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:33.153785 9117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:33.153961 9117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:33.161206 9117 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 22:59:33.161316 9117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:33.161363 9117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:33.161448 9117 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 22:59:33.161511 9117 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 22:59:33.171518 9117 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:39019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:36895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.62.keytab
--principal=oryx/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:36895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:33.172863 9117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:33.174196 9117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:33.186600 9125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:33.186666 9123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:33.186542 9122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:33.187289 9117 server_base.cc:1034] running on GCE node
I20260620 22:59:33.188086 9117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:33.189522 9117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:33.190758 9117 hybrid_clock.cc:648] HybridClock initialized: now 1781996373190708 us; error 55 us; skew 500 ppm
I20260620 22:59:33.191165 9117 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:33 dist-test-slave-f63b krb5kdc[9104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996373, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:33.194842 9117 init.cc:377] Logged in from keytab as oryx/127.2.237.62@KRBTEST.COM (short username oryx)
I20260620 22:59:33.196341 9117 webserver.cc:460] Webserver started at http://127.2.237.62:40805/ using document root <none> and password file <none>
I20260620 22:59:33.197088 9117 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:33.197250 9117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:33.197579 9117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:33.200755 9117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "988aa53a4d75433ba11c4b68df409c91"
format_stamp: "Formatted at 2026-06-20 22:59:33 on dist-test-slave-f63b"
server_key: "390aeb126b0bd87f3f5a38c3341506f8"
server_key_iv: "ab00c05b7e4fd91ce20651731485fac7"
server_key_version: "encryptionkey@0"
I20260620 22:59:33.201586 9117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "988aa53a4d75433ba11c4b68df409c91"
format_stamp: "Formatted at 2026-06-20 22:59:33 on dist-test-slave-f63b"
server_key: "390aeb126b0bd87f3f5a38c3341506f8"
server_key_iv: "ab00c05b7e4fd91ce20651731485fac7"
server_key_version: "encryptionkey@0"
I20260620 22:59:33.206944 9117 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260620 22:59:33.210448 9132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:33.211971 9117 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 22:59:33.212175 9117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "988aa53a4d75433ba11c4b68df409c91"
format_stamp: "Formatted at 2026-06-20 22:59:33 on dist-test-slave-f63b"
server_key: "390aeb126b0bd87f3f5a38c3341506f8"
server_key_iv: "ab00c05b7e4fd91ce20651731485fac7"
server_key_version: "encryptionkey@0"
I20260620 22:59:33.212373 9117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:33.235675 9117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:33.240139 9117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:33.240425 9117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:33.258646 9117 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:36895
I20260620 22:59:33.258682 9183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:36895 every 8 connection(s)
I20260620 22:59:33.260094 9117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 22:59:33.263795 9184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:33.270471 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 9117
I20260620 22:59:33.270735 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 22:59:33.271522 2996 external_mini_cluster.cc:1428] Setting key 1320c1384121f255157012e91e3f2cd2
I20260620 22:59:33.275157 9184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91: Bootstrap starting.
I20260620 22:59:33.278290 9184 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:33.279404 9184 log.cc:824] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91: Log is configured to *not* fsync() on all Append() calls
I20260620 22:59:33.282732 9184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91: No bootstrap required, opened a new log
Jun 20 22:59:33 dist-test-slave-f63b krb5kdc[9104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM
I20260620 22:59:33.293437 9184 raft_consensus.cc:348] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } }
I20260620 22:59:33.293951 9184 raft_consensus.cc:374] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:33.294061 9184 raft_consensus.cc:729] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 988aa53a4d75433ba11c4b68df409c91, State: Initialized, Role: FOLLOWER
I20260620 22:59:33.294749 9184 consensus_queue.cc:260] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } }
I20260620 22:59:33.295053 9184 raft_consensus.cc:388] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 22:59:33.295236 9184 raft_consensus.cc:482] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 22:59:33.295445 9184 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:33.296996 9187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:33.273750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37946 (local address 127.2.237.62:36895)
0620 22:59:33.274510 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:33.274522 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:33.274563 (+ 41us) server_negotiation.cc:407] Connection header received
0620 22:59:33.276426 (+ 1863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:33.276450 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:33.276862 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:33.277244 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:33.278760 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:33.280407 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:33.282193 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:33.283033 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:33.283190 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:33.287888 (+ 4698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:33.287899 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:33.287928 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:33.287979 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:33.292141 (+ 4162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:33.292864 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:33.292873 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:33.292882 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:33.293099 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:33.293507 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:33.293513 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:33.293515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:33.294252 (+ 737us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:33.294542 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:33.295080 (+ 538us) server_negotiation.cc:299] Negotiation successful
0620 22:59:33.295571 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":340,"threads_started":1}
I20260620 22:59:33.298307 9184 raft_consensus.cc:504] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } }
I20260620 22:59:33.298884 9184 leader_election.cc:302] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 988aa53a4d75433ba11c4b68df409c91; no voters:
I20260620 22:59:33.299489 9184 leader_election.cc:288] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 22:59:33.299639 9189 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:59:33.300680 9189 raft_consensus.cc:686] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [term 1 LEADER]: Becoming Leader. State: Replica: 988aa53a4d75433ba11c4b68df409c91, State: Running, Role: LEADER
I20260620 22:59:33.301182 9189 consensus_queue.cc:237] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } }
I20260620 22:59:33.301532 9184 sys_catalog.cc:564] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 22:59:33.303959 9191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 988aa53a4d75433ba11c4b68df409c91. Latest consensus state: current_term: 1 leader_uuid: "988aa53a4d75433ba11c4b68df409c91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } }
I20260620 22:59:33.304301 9191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:33.304157 9190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "988aa53a4d75433ba11c4b68df409c91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } }
I20260620 22:59:33.304577 9190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:33.306881 9196 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 22:59:33.311437 9196 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 22:59:33.321141 9196 catalog_manager.cc:1347] Generated new cluster ID: 77a818f56f2d4bb9a22550a45993c572
I20260620 22:59:33.321274 9196 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 22:59:33.337664 9196 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 22:59:33.339071 9196 catalog_manager.cc:1504] Loading token signing keys...
I20260620 22:59:33.349689 9196 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91: Generated new TSK 0
I20260620 22:59:33.350898 9196 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.2.237.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.237.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.237.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.1.keytab.
Entry for principal oryx/127.2.237.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.1.keytab.
I20260620 22:59:33.414475 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:0
--local_ip_for_outbound_sockets=127.2.237.1
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:36895
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.1.keytab
--principal=oryx/127.2.237.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:39019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:33.744700 9210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:33.745044 9210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:33.745218 9210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:33.753072 9210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:33.753247 9210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:33.753423 9210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 22:59:33.763227 9210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:39019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.1.keytab
--principal=oryx/127.2.237.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:36895
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:33.764572 9210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:33.765962 9210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:33.779656 9215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:33.779683 9218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:33.779735 9216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:33.780637 9210 server_base.cc:1034] running on GCE node
I20260620 22:59:33.781452 9210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:33.782424 9210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:33.783658 9210 hybrid_clock.cc:648] HybridClock initialized: now 1781996373783599 us; error 54 us; skew 500 ppm
I20260620 22:59:33.783984 9210 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:33 dist-test-slave-f63b krb5kdc[9104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996373, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:33.787122 9210 init.cc:377] Logged in from keytab as oryx/127.2.237.1@KRBTEST.COM (short username oryx)
I20260620 22:59:33.788595 9210 webserver.cc:460] Webserver started at http://127.2.237.1:35785/ using document root <none> and password file <none>
I20260620 22:59:33.789299 9210 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:33.789440 9210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:33.789746 9210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:33.792647 9210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/data/instance:
uuid: "8154a02194f741cbb6596035cec6fcf0"
format_stamp: "Formatted at 2026-06-20 22:59:33 on dist-test-slave-f63b"
server_key: "abc5e1d34256f74cdfb5abcdb9124685"
server_key_iv: "e6f7edc3087bfc7db5d629c515c5abd1"
server_key_version: "encryptionkey@0"
I20260620 22:59:33.793388 9210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance:
uuid: "8154a02194f741cbb6596035cec6fcf0"
format_stamp: "Formatted at 2026-06-20 22:59:33 on dist-test-slave-f63b"
server_key: "abc5e1d34256f74cdfb5abcdb9124685"
server_key_iv: "e6f7edc3087bfc7db5d629c515c5abd1"
server_key_version: "encryptionkey@0"
I20260620 22:59:33.799110 9210 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260620 22:59:33.802729 9225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:33.804112 9210 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 22:59:33.804283 9210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "8154a02194f741cbb6596035cec6fcf0"
format_stamp: "Formatted at 2026-06-20 22:59:33 on dist-test-slave-f63b"
server_key: "abc5e1d34256f74cdfb5abcdb9124685"
server_key_iv: "e6f7edc3087bfc7db5d629c515c5abd1"
server_key_version: "encryptionkey@0"
I20260620 22:59:33.804488 9210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:33.821238 9210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:33.826119 9210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:33.826408 9210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:33.827409 9210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:59:33.828747 9210 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:33.828848 9210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:33.829008 9210 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:33.829080 9210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:33.861378 9210 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:41217
I20260620 22:59:33.861420 9337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:41217 every 8 connection(s)
I20260620 22:59:33.862785 9210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 22:59:33.866930 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 9210
I20260620 22:59:33.867146 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance
I20260620 22:59:33.867969 2996 external_mini_cluster.cc:1428] Setting key 81efcbf9687cdd66f59f81e793386caf
Jun 20 22:59:33 dist-test-slave-f63b krb5kdc[9104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996373, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM
I20260620 22:59:33.886176 9340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:33.865413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:37461 (local address 127.2.237.62:36895)
0620 22:59:33.865834 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:33.865839 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:33.867157 (+ 1318us) server_negotiation.cc:407] Connection header received
0620 22:59:33.869360 (+ 2203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:33.869365 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:33.869454 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:33.869572 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:33.871652 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:33.872365 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:33.874914 (+ 2549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:33.875323 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:33.875423 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:33.879665 (+ 4242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:33.879670 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:33.879673 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:33.879706 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:33.882340 (+ 2634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:33.883362 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:33.883367 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:33.883370 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:33.883503 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:33.884135 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:33.884140 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:33.884142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:33.884428 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:33.884593 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:33.885651 (+ 1058us) server_negotiation.cc:299] Negotiation successful
0620 22:59:33.885915 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":225,"threads_started":1}
I20260620 22:59:33.887475 9341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:33.866061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.2.237.1:37461)
0620 22:59:33.866940 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:33.866997 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:33.869096 (+ 2099us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:33.869823 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:33.869855 (+ 32us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:33.870613 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:33.871458 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:33.871487 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:33.872533 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:33.872541 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:33.874748 (+ 2207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:33.874766 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:33.875553 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:33.875562 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:33.875909 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:33.876676 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:33.876736 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:33.879418 (+ 2682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:33.882489 (+ 3071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:33.882501 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:33.882522 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:33.883194 (+ 672us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:33.883694 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:33.883702 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:33.883708 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:33.883969 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:33.884599 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:33.884610 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:33.885137 (+ 527us) client_negotiation.cc:769] Sending connection context
0620 22:59:33.885464 (+ 327us) client_negotiation.cc:240] Negotiation successful
0620 22:59:33.886114 (+ 650us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":299,"threads_started":1}
I20260620 22:59:33.889274 9338 heartbeater.cc:347] Connected to a master server at 127.2.237.62:36895
I20260620 22:59:33.889622 9338 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for oryx/127.2.237.2@KRBTEST.COM; defaulting to no policy
I20260620 22:59:33.890496 9338 heartbeater.cc:511] Master 127.2.237.62:36895 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.237.2@KRBTEST.COM" created.
I20260620 22:59:33.893107 9149 ts_manager.cc:194] Registered new tserver with Master: 8154a02194f741cbb6596035cec6fcf0 (127.2.237.1:41217)
I20260620 22:59:33.894654 9149 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.237.1@KRBTEST.COM'} at 127.2.237.1:37461
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.237.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.2.keytab.
Entry for principal oryx/127.2.237.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.2.keytab.
I20260620 22:59:33.920017 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:0
--local_ip_for_outbound_sockets=127.2.237.2
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:36895
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.2.keytab
--principal=oryx/127.2.237.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:39019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:34.246749 9344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:34.247095 9344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:34.247278 9344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:34.254475 9344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:34.254693 9344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:34.254881 9344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 22:59:34.268962 9344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:39019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.2.keytab
--principal=oryx/127.2.237.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:36895
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:34.270494 9344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:34.272377 9344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:34.288851 9349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:34.288812 9352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:34.288890 9350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:34.289258 9344 server_base.cc:1034] running on GCE node
I20260620 22:59:34.290089 9344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:34.291124 9344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:34.292327 9344 hybrid_clock.cc:648] HybridClock initialized: now 1781996374292258 us; error 61 us; skew 500 ppm
I20260620 22:59:34.292678 9344 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:34 dist-test-slave-f63b krb5kdc[9104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996374, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:34.299743 9344 init.cc:377] Logged in from keytab as oryx/127.2.237.2@KRBTEST.COM (short username oryx)
I20260620 22:59:34.301394 9344 webserver.cc:460] Webserver started at http://127.2.237.2:38341/ using document root <none> and password file <none>
I20260620 22:59:34.302265 9344 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:34.302413 9344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:34.302826 9344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:34.305830 9344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/data/instance:
uuid: "d3ffb93de8d841159da80de89f7171e4"
format_stamp: "Formatted at 2026-06-20 22:59:34 on dist-test-slave-f63b"
server_key: "64a2476ffc991825fa208bfd04090737"
server_key_iv: "170d7e6d4b98a2e5989ee0a198d97b8d"
server_key_version: "encryptionkey@0"
I20260620 22:59:34.306802 9344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance:
uuid: "d3ffb93de8d841159da80de89f7171e4"
format_stamp: "Formatted at 2026-06-20 22:59:34 on dist-test-slave-f63b"
server_key: "64a2476ffc991825fa208bfd04090737"
server_key_iv: "170d7e6d4b98a2e5989ee0a198d97b8d"
server_key_version: "encryptionkey@0"
I20260620 22:59:34.313016 9344 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260620 22:59:34.316991 9359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:34.318509 9344 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 22:59:34.318720 9344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "d3ffb93de8d841159da80de89f7171e4"
format_stamp: "Formatted at 2026-06-20 22:59:34 on dist-test-slave-f63b"
server_key: "64a2476ffc991825fa208bfd04090737"
server_key_iv: "170d7e6d4b98a2e5989ee0a198d97b8d"
server_key_version: "encryptionkey@0"
I20260620 22:59:34.318930 9344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:34.340073 9344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:34.344340 9344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:34.344652 9344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:34.345628 9344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:59:34.346992 9344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:34.347095 9344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:34.347236 9344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:34.347306 9344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:34.381186 9344 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:33799
I20260620 22:59:34.381234 9471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:33799 every 8 connection(s)
I20260620 22:59:34.382493 9344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 22:59:34.392089 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 9344
I20260620 22:59:34.392269 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance
I20260620 22:59:34.392807 2996 external_mini_cluster.cc:1428] Setting key 4e886d45d6b3320fd00aa1d72e232d1d
Jun 20 22:59:34 dist-test-slave-f63b krb5kdc[9104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996374, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM
I20260620 22:59:34.405012 9340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:34.385077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:58805 (local address 127.2.237.62:36895)
0620 22:59:34.385249 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:34.385254 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:34.386497 (+ 1243us) server_negotiation.cc:407] Connection header received
0620 22:59:34.388491 (+ 1994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:34.388497 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:34.388592 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:34.388700 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:34.390193 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:34.390969 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:34.393058 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:34.393616 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:34.393728 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:34.397677 (+ 3949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:34.397686 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:34.397695 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:34.397738 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:34.401368 (+ 3630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:34.402348 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:34.402352 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:34.402354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:34.402475 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:34.403033 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:34.403041 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:34.403046 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:34.403430 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:34.403580 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:34.404296 (+ 716us) server_negotiation.cc:299] Negotiation successful
0620 22:59:34.404555 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 22:59:34.405663 9474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:34.385578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.2.237.2:58805)
0620 22:59:34.386326 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:34.386366 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:34.388195 (+ 1829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:34.388926 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:34.388935 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:34.389462 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:34.390017 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:34.390035 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:34.391141 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:34.391145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:34.392917 (+ 1772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:34.392934 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:34.393841 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:34.393854 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:34.394049 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:34.394735 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:34.394773 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:34.397438 (+ 2665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:34.401515 (+ 4077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:34.401527 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:34.401548 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:34.402237 (+ 689us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:34.402619 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:34.402626 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:34.402631 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:34.402892 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:34.403552 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:34.403558 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:34.403915 (+ 357us) client_negotiation.cc:769] Sending connection context
0620 22:59:34.404148 (+ 233us) client_negotiation.cc:240] Negotiation successful
0620 22:59:34.404531 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":273,"threads_started":1}
I20260620 22:59:34.407244 9472 heartbeater.cc:347] Connected to a master server at 127.2.237.62:36895
I20260620 22:59:34.407572 9472 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:34.408459 9472 heartbeater.cc:511] Master 127.2.237.62:36895 requested a full tablet report, sending...
I20260620 22:59:34.409994 9149 ts_manager.cc:194] Registered new tserver with Master: d3ffb93de8d841159da80de89f7171e4 (127.2.237.2:33799)
I20260620 22:59:34.410858 9149 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.237.2@KRBTEST.COM'} at 127.2.237.2:58805
WARNING: no policy specified for oryx/127.2.237.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.237.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.237.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.3.keytab.
Entry for principal oryx/127.2.237.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.3.keytab.
I20260620 22:59:34.441483 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:0
--local_ip_for_outbound_sockets=127.2.237.3
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:36895
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.3.keytab
--principal=oryx/127.2.237.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:39019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:34.763340 9477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:34.763679 9477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:34.763856 9477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:34.771348 9477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:34.771513 9477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:34.771750 9477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 22:59:34.781771 9477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:39019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.3.keytab
--principal=oryx/127.2.237.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:36895
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:34.783038 9477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:34.784296 9477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:34.796712 9483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:34.796785 9482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:34.797135 9485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:34.797765 9477 server_base.cc:1034] running on GCE node
I20260620 22:59:34.798293 9477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:34.799245 9477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:34.800469 9477 hybrid_clock.cc:648] HybridClock initialized: now 1781996374800418 us; error 63 us; skew 500 ppm
I20260620 22:59:34.800788 9477 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:34 dist-test-slave-f63b krb5kdc[9104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996374, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:34.804003 9477 init.cc:377] Logged in from keytab as oryx/127.2.237.3@KRBTEST.COM (short username oryx)
I20260620 22:59:34.805418 9477 webserver.cc:460] Webserver started at http://127.2.237.3:43921/ using document root <none> and password file <none>
I20260620 22:59:34.806125 9477 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:34.806262 9477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:34.806552 9477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:34.809705 9477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/data/instance:
uuid: "3846f009be844b4ab3849d5ad442e6cb"
format_stamp: "Formatted at 2026-06-20 22:59:34 on dist-test-slave-f63b"
server_key: "e9c50b728a8bca74a40f96f9bcdc647f"
server_key_iv: "8a4f5a2364110dc0e423f95efeb403be"
server_key_version: "encryptionkey@0"
I20260620 22:59:34.810431 9477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance:
uuid: "3846f009be844b4ab3849d5ad442e6cb"
format_stamp: "Formatted at 2026-06-20 22:59:34 on dist-test-slave-f63b"
server_key: "e9c50b728a8bca74a40f96f9bcdc647f"
server_key_iv: "8a4f5a2364110dc0e423f95efeb403be"
server_key_version: "encryptionkey@0"
I20260620 22:59:34.815672 9477 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260620 22:59:34.819065 9492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:34.820370 9477 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260620 22:59:34.820523 9477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "3846f009be844b4ab3849d5ad442e6cb"
format_stamp: "Formatted at 2026-06-20 22:59:34 on dist-test-slave-f63b"
server_key: "e9c50b728a8bca74a40f96f9bcdc647f"
server_key_iv: "8a4f5a2364110dc0e423f95efeb403be"
server_key_version: "encryptionkey@0"
I20260620 22:59:34.820690 9477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:34.840384 9477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:34.844695 9477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:34.844988 9477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:34.846203 9477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:59:34.847615 9477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:34.847685 9477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:34.847776 9477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:34.847818 9477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:34.880136 9477 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:34669
I20260620 22:59:34.880208 9604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:34669 every 8 connection(s)
I20260620 22:59:34.881429 9477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 22:59:34.890951 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 9477
I20260620 22:59:34.891112 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance
I20260620 22:59:34.891562 2996 external_mini_cluster.cc:1428] Setting key c3ef2158a0a1e05e8e25bcd396f64e55
Jun 20 22:59:34 dist-test-slave-f63b krb5kdc[9104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996374, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM
I20260620 22:59:34.897578 9338 heartbeater.cc:503] Master 127.2.237.62:36895 was elected leader, sending a full tablet report...
I20260620 22:59:34.901626 9340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:34.884282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:43569 (local address 127.2.237.62:36895)
0620 22:59:34.884482 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:34.884489 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:34.885722 (+ 1233us) server_negotiation.cc:407] Connection header received
0620 22:59:34.887503 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:34.887507 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:34.887585 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:34.887671 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:34.889212 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:34.889966 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:34.891620 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:34.892193 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:34.892329 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:34.896260 (+ 3931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:34.896265 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:34.896268 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:34.896297 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:34.899047 (+ 2750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:34.899702 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:34.899711 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:34.899713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:34.899830 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:34.900206 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:34.900210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:34.900213 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:34.900452 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:34.900551 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:34.901164 (+ 613us) server_negotiation.cc:299] Negotiation successful
0620 22:59:34.901376 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260620 22:59:34.902662 9607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:34.884832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.2.237.3:43569)
0620 22:59:34.885560 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:34.885610 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:34.887252 (+ 1642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:34.887929 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:34.887937 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:34.888470 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:34.889060 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:34.889078 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:34.890084 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:34.890087 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:34.891454 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:34.891466 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:34.892423 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:34.892428 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:34.892749 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:34.893508 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:34.893566 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:34.896084 (+ 2518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:34.899165 (+ 3081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:34.899179 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:34.899190 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:34.899592 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:34.899922 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:34.899925 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:34.899927 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:34.900114 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:34.900555 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:34.900563 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:34.900857 (+ 294us) client_negotiation.cc:769] Sending connection context
0620 22:59:34.901088 (+ 231us) client_negotiation.cc:240] Negotiation successful
0620 22:59:34.901509 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":273,"threads_started":1}
I20260620 22:59:34.904013 9605 heartbeater.cc:347] Connected to a master server at 127.2.237.62:36895
I20260620 22:59:34.904296 9605 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:34.905014 9605 heartbeater.cc:511] Master 127.2.237.62:36895 requested a full tablet report, sending...
I20260620 22:59:34.906780 9149 ts_manager.cc:194] Registered new tserver with Master: 3846f009be844b4ab3849d5ad442e6cb (127.2.237.3:34669)
I20260620 22:59:34.907680 9149 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.237.3@KRBTEST.COM'} at 127.2.237.3:43569
I20260620 22:59:34.919159 2996 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.2.237.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.237.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.237.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.61.keytab.
Entry for principal oryx/127.2.237.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.61.keytab.
I20260620 22:59:34.966141 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.61:38995
--webserver_interface=127.2.237.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.61.keytab
--principal=oryx/127.2.237.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:39019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:36895,127.2.237.61:38995
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:35.283380 9610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:35.283658 9610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:35.283851 9610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:35.290928 9610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 22:59:35.291026 9610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:35.291133 9610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:35.291221 9610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 22:59:35.291301 9610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 22:59:35.301047 9610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:39019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:36895,127.2.237.61:38995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.61.keytab
--principal=oryx/127.2.237.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.61:38995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.237.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
W20260620 22:59:35.301450 9610 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.2.237.62:36895,127.2.237.61:38995'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260620 22:59:35.302538 9610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:35.303941 9610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:35.315481 9615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:35.315424 9616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:35.315459 9618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:35.316313 9610 server_base.cc:1034] running on GCE node
I20260620 22:59:35.317173 9610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:35.318650 9610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:35.319871 9610 hybrid_clock.cc:648] HybridClock initialized: now 1781996375319797 us; error 52 us; skew 500 ppm
I20260620 22:59:35.320185 9610 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:35 dist-test-slave-f63b krb5kdc[9104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996375, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:35.323319 9610 init.cc:377] Logged in from keytab as oryx/127.2.237.61@KRBTEST.COM (short username oryx)
I20260620 22:59:35.324774 9610 webserver.cc:460] Webserver started at http://127.2.237.61:37753/ using document root <none> and password file <none>
I20260620 22:59:35.325551 9610 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:35.325641 9610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:35.325965 9610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:35.328887 9610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-1/data/instance:
uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa"
format_stamp: "Formatted at 2026-06-20 22:59:35 on dist-test-slave-f63b"
server_key: "0c9e8e3a9ee82d76ac7a69f1962ba821"
server_key_iv: "d79f8813014c7406356563f3680fd2e5"
server_key_version: "encryptionkey@0"
I20260620 22:59:35.329746 9610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-1/wal/instance:
uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa"
format_stamp: "Formatted at 2026-06-20 22:59:35 on dist-test-slave-f63b"
server_key: "0c9e8e3a9ee82d76ac7a69f1962ba821"
server_key_iv: "d79f8813014c7406356563f3680fd2e5"
server_key_version: "encryptionkey@0"
I20260620 22:59:35.335984 9610 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.001s
I20260620 22:59:35.339413 9625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:35.340700 9610 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 22:59:35.340865 9610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-1/wal
uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa"
format_stamp: "Formatted at 2026-06-20 22:59:35 on dist-test-slave-f63b"
server_key: "0c9e8e3a9ee82d76ac7a69f1962ba821"
server_key_iv: "d79f8813014c7406356563f3680fd2e5"
server_key_version: "encryptionkey@0"
I20260620 22:59:35.341084 9610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:35.359236 9610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:35.363502 9610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:35.363770 9610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:35.385275 9610 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.61:38995
I20260620 22:59:35.385298 9676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.61:38995 every 8 connection(s)
I20260620 22:59:35.386785 9610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-1/data/info.pb
I20260620 22:59:35.390420 9677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:35.396129 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 9610
I20260620 22:59:35.397455 9677 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 22:59:35 dist-test-slave-f63b krb5kdc[9104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996375, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.61@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM
Jun 20 22:59:35 dist-test-slave-f63b krb5kdc[9104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.237.61@KRBTEST.COM
I20260620 22:59:35.414286 9472 heartbeater.cc:503] Master 127.2.237.62:36895 was elected leader, sending a full tablet report...
I20260620 22:59:35.418081 9340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:35.400230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37948 (local address 127.2.237.62:36895)
0620 22:59:35.400462 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:35.400468 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:35.401051 (+ 583us) server_negotiation.cc:407] Connection header received
0620 22:59:35.402108 (+ 1057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:35.402112 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:35.402176 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:35.402261 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:35.403660 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:35.404580 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:35.406764 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:35.407483 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:35.407650 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:35.411475 (+ 3825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:35.411482 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:35.411487 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:35.411524 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:35.414379 (+ 2855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:35.415336 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:35.415356 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:35.415361 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:35.415511 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:35.416212 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:35.416217 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:35.416220 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:35.416595 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:35.416748 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:35.417549 (+ 801us) server_negotiation.cc:299] Negotiation successful
0620 22:59:35.417759 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260620 22:59:35.419185 9681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:35.400175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.0.0.1:37948)
0620 22:59:35.400882 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:35.400930 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:35.401841 (+ 911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:35.402344 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:35.402350 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:35.403154 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:35.403494 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:35.403514 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:35.404699 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:35.404718 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:35.406641 (+ 1923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:35.406653 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:35.407730 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:35.407735 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:35.407930 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:35.408715 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:35.408747 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:35.411290 (+ 2543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:35.414535 (+ 3245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:35.414546 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:35.414557 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:35.415219 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:35.415615 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:35.415620 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:35.415623 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:35.415897 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:35.416720 (+ 823us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:35.416728 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:35.417105 (+ 377us) client_negotiation.cc:769] Sending connection context
0620 22:59:35.417465 (+ 360us) client_negotiation.cc:240] Negotiation successful
0620 22:59:35.417947 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":111,"thread_start_us":270,"threads_started":1}
I20260620 22:59:35.419282 9680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:35.398885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40498 (local address 127.2.237.61:38995)
0620 22:59:35.399589 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:35.399599 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:35.399634 (+ 35us) server_negotiation.cc:407] Connection header received
0620 22:59:35.401841 (+ 2207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:35.401868 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:35.402153 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:35.402372 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:35.404196 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:35.405893 (+ 1697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:35.407728 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:35.408364 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:35.408554 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:35.412936 (+ 4382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:35.412943 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:35.412955 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:35.412982 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:35.416293 (+ 3311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:35.416930 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:35.416938 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:35.416941 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:35.417072 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:35.417559 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:35.417568 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:35.417570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:35.417866 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:35.418164 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:35.418701 (+ 537us) server_negotiation.cc:299] Negotiation successful
0620 22:59:35.418925 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":524,"thread_start_us":315,"threads_started":1}
I20260620 22:59:35.422370 9677 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 22:59:35 dist-test-slave-f63b krb5kdc[9104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996375, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.61@KRBTEST.COM for oryx/127.2.237.61@KRBTEST.COM
I20260620 22:59:35.434223 9681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:35.423256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:38995 (local address 127.0.0.1:40500)
0620 22:59:35.423418 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:35.423437 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:35.423528 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:35.423765 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:35.423768 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:35.424133 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:35.424313 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:35.424325 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:35.425561 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:35.425580 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:35.426415 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:35.426425 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:35.427115 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:35.427120 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:35.427270 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:35.427728 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:35.427743 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:35.429580 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:35.432340 (+ 2760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:35.432344 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:35.432346 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:35.432738 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:35.433040 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:35.433043 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:35.433045 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:35.433158 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:35.433564 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:35.433569 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:35.433691 (+ 122us) client_negotiation.cc:769] Sending connection context
0620 22:59:35.433795 (+ 104us) client_negotiation.cc:240] Negotiation successful
0620 22:59:35.433983 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260620 22:59:35.434356 9680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:35.423265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40500 (local address 127.2.237.61:38995)
0620 22:59:35.423423 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:35.423426 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:35.423479 (+ 53us) server_negotiation.cc:407] Connection header received
0620 22:59:35.423593 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:35.423596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:35.423643 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:35.423733 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:35.424474 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:35.425415 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:35.426520 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:35.426971 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:35.427068 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:35.429750 (+ 2682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:35.429754 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:35.429757 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:35.429782 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:35.432223 (+ 2441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:35.432829 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:35.432832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:35.432834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:35.432947 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:35.433240 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:35.433243 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:35.433245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:35.433473 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:35.433571 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:35.433869 (+ 298us) server_negotiation.cc:299] Negotiation successful
0620 22:59:35.434088 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":58}
I20260620 22:59:35.438884 9677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa: Bootstrap starting.
I20260620 22:59:35.441833 9677 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:35.442878 9677 log.cc:824] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa: Log is configured to *not* fsync() on all Append() calls
I20260620 22:59:35.445439 9677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa: No bootstrap required, opened a new log
I20260620 22:59:35.452391 9677 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } }
I20260620 22:59:35.452679 9677 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:35.452783 9677 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65dcb1d2184d4d41bb04de52e7e5e2aa, State: Initialized, Role: FOLLOWER
I20260620 22:59:35.453316 9677 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } }
I20260620 22:59:35.454099 9683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } } }
I20260620 22:59:35.454381 9683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [sys.catalog]: This master's current role is: FOLLOWER
I20260620 22:59:35.455148 9677 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [sys.catalog]: configured and running, proceeding with master startup.
W20260620 22:59:35.463627 9694 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 22:59:35.463761 9694 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 22:59:35.465548 9610 master_runner.cc:428] Detected that this master 65dcb1d2184d4d41bb04de52e7e5e2aa is joining an existing cluster
I20260620 22:59:35.465705 9610 master_runner.cc:432] Initiating AddMaster RPC to add 127.2.237.61:38995
I20260620 22:59:35.486343 9701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:35.469634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:38995 (local address 127.0.0.1:40512)
0620 22:59:35.471069 (+ 1435us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:35.471092 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:35.471209 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:35.471575 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:35.471580 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:35.472288 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:35.472620 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:35.472633 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:35.474102 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:35.474107 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:35.475930 (+ 1823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:35.475947 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:35.477423 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:35.477430 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:35.477645 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:35.478352 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:35.478378 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:35.479667 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:35.483640 (+ 3973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:35.483648 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:35.483655 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:35.484104 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:35.484453 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:35.484457 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:35.484461 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:35.484586 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:35.485119 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:35.485126 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:35.485258 (+ 132us) client_negotiation.cc:769] Sending connection context
0620 22:59:35.485452 (+ 194us) client_negotiation.cc:240] Negotiation successful
0620 22:59:35.485788 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1305,"mutex_wait_us":27,"spinlock_wait_cycles":20352,"thread_start_us":1103,"threads_started":1}
I20260620 22:59:35.486385 9680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:35.469543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40512 (local address 127.2.237.61:38995)
0620 22:59:35.469686 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:35.469693 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:35.471180 (+ 1487us) server_negotiation.cc:407] Connection header received
0620 22:59:35.471325 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:35.471331 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:35.471413 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:35.471527 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:35.472756 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:35.473874 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:35.476107 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:35.476986 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:35.477118 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:35.479848 (+ 2730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:35.479857 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:35.479865 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:35.479910 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:35.483462 (+ 3552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:35.484227 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:35.484233 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:35.484236 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:35.484348 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:35.484691 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:35.484696 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:35.484699 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:35.484992 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:35.485146 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:35.485552 (+ 406us) server_negotiation.cc:299] Negotiation successful
0620 22:59:35.485959 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":168,"server-negotiator.queue_time_us":37}
I20260620 22:59:35.488126 9340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:35.469247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37952 (local address 127.2.237.62:36895)
0620 22:59:35.469462 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:35.469470 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:35.470387 (+ 917us) server_negotiation.cc:407] Connection header received
0620 22:59:35.470609 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:35.470617 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:35.470709 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:35.470810 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:35.472753 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:35.473874 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:35.476802 (+ 2928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:35.477473 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:35.477572 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:35.480596 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:35.480862 (+ 266us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:35.480870 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:35.480914 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:35.484633 (+ 3719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:35.485416 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:35.485423 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:35.485428 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:35.485592 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:35.486035 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:35.486040 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:35.486044 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:35.486401 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:35.486774 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:35.487539 (+ 765us) server_negotiation.cc:299] Negotiation successful
0620 22:59:35.487776 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260620 22:59:35.488134 9700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:35.469582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.0.0.1:37952)
0620 22:59:35.470286 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:35.470309 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:35.470455 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:35.471001 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:35.471006 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:35.471850 (+ 844us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:35.472568 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:35.472585 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:35.474721 (+ 2136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:35.474726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:35.476625 (+ 1899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:35.476639 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:35.477855 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:35.477862 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:35.478416 (+ 554us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:35.479064 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:35.479092 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:35.480385 (+ 1293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:35.484787 (+ 4402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:35.484795 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:35.484805 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:35.485294 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:35.485732 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:35.485736 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:35.485740 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:35.485909 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:35.486925 (+ 1016us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:35.486928 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:35.487182 (+ 254us) client_negotiation.cc:769] Sending connection context
0620 22:59:35.487365 (+ 183us) client_negotiation.cc:240] Negotiation successful
0620 22:59:35.487669 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":57,"thread_start_us":435,"threads_started":1}
Jun 20 22:59:35 dist-test-slave-f63b krb5kdc[9104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996373, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.62@KRBTEST.COM for oryx/127.2.237.61@KRBTEST.COM
I20260620 22:59:35.507915 9680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:35.493877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40520 (local address 127.2.237.61:38995)
0620 22:59:35.494105 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:35.494113 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:35.494663 (+ 550us) server_negotiation.cc:407] Connection header received
0620 22:59:35.494931 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:35.494935 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:35.495018 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:35.495110 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:35.496359 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:35.497126 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:35.498409 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:35.498946 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:35.499088 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:35.502232 (+ 3144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:35.502237 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:35.502240 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:35.502267 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:35.505126 (+ 2859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:35.505800 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:35.505806 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:35.505810 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:35.505943 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:35.506300 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:35.506307 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:35.506310 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:35.506722 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:35.506881 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:35.507314 (+ 433us) server_negotiation.cc:299] Negotiation successful
0620 22:59:35.507549 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
WARNING: no policy specified for oryx/127.2.237.60@KRBTEST.COM; defaulting to no policy
I20260620 22:59:35.508186 9703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:35.493839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:38995 (local address 127.0.0.1:40520)
0620 22:59:35.494511 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:35.494561 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:35.494763 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:35.495175 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:35.495182 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:35.495931 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:35.496189 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:35.496212 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:35.497257 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:35.497264 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:35.498274 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:35.498286 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:35.499114 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:35.499120 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:35.499306 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:35.499878 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:35.499893 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:35.502070 (+ 2177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:35.505254 (+ 3184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:35.505268 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:35.505276 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:35.505685 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:35.506055 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:35.506061 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:35.506065 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:35.506198 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:35.506860 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:35.506869 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:35.507054 (+ 185us) client_negotiation.cc:769] Sending connection context
0620 22:59:35.507237 (+ 183us) client_negotiation.cc:240] Negotiation successful
0620 22:59:35.507624 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":291,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.237.60@KRBTEST.COM" created.
I20260620 22:59:35.510766 9149 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.2.237.61:38995: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa"
member_type: NON_VOTER
last_known_addr {
host: "127.2.237.61"
port: 38995
}
attrs {
promote: true
}
}
dest_uuid: "988aa53a4d75433ba11c4b68df409c91"
cas_config_opid_index: -1
I20260620 22:59:35.511744 9149 consensus_queue.cc:237] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: NON_VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: true } }
I20260620 22:59:35.520121 9652 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:35.520599 9706 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 65dcb1d2184d4d41bb04de52e7e5e2aa (127.2.237.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: NON_VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: true } } }
I20260620 22:59:35.524150 9652 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 1 FOLLOWER]: Refusing update from remote peer 988aa53a4d75433ba11c4b68df409c91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260620 22:59:35.525072 9706 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.2.237.61:38995
I20260620 22:59:35.525739 9610 master.cc:537] Master@127.2.237.61:38995 shutting down...
I20260620 22:59:35.526506 9705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "988aa53a4d75433ba11c4b68df409c91" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: NON_VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: true } } }
I20260620 22:59:35.526896 9705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:35.528703 9706 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: NON_VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 22:59:35.538661 9134 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260620 22:59:35.543860 9610 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 1 FOLLOWER]: Raft consensus shutting down.
W20260620 22:59:35.544353 9134 consensus_peers.cc:487] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 -> Peer 65dcb1d2184d4d41bb04de52e7e5e2aa (127.2.237.61:38995): Couldn't send request to peer 65dcb1d2184d4d41bb04de52e7e5e2aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260620 22:59:35.544360 9610 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:35.544606 9610 tablet_replica.cc:330] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.237.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.60.keytab.
Entry for principal oryx/127.2.237.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.60.keytab.
I20260620 22:59:35.557744 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.60:42911
--webserver_interface=127.2.237.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.60.keytab
--principal=oryx/127.2.237.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:39019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:36895,127.2.237.61:38995,127.2.237.60:42911
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 22:59:35.561425 9610 master.cc:552] Master@127.2.237.61:38995 shutdown complete.
I20260620 22:59:35.561546 9610 master_runner.cc:305] Clearing existing system tablet
I20260620 22:59:35.562005 9610 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:35.562112 9610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:35.565060 9610 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260620 22:59:35.566675 9713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:35.567101 9610 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:35.567365 9610 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260620 22:59:35.567451 9610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-1/wal
uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa"
format_stamp: "Formatted at 2026-06-20 22:59:35 on dist-test-slave-f63b"
server_key: "0c9e8e3a9ee82d76ac7a69f1962ba821"
server_key_iv: "d79f8813014c7406356563f3680fd2e5"
server_key_version: "encryptionkey@0"
I20260620 22:59:35.572507 9610 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:35.579375 9610 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260620 22:59:35.579553 9610 log.cc:1196] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260620 22:59:35.580021 9610 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa: Deleting consensus metadata
I20260620 22:59:35.580279 9610 master_runner.cc:315] Copying system tablet from 127.2.237.62:36895
I20260620 22:59:35.582818 9610 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa: tablet copy: Beginning tablet copy session from remote peer at address 127.2.237.62:36895
I20260620 22:59:35.600608 9340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:35.583711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37966 (local address 127.2.237.62:36895)
0620 22:59:35.583890 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:35.583896 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:35.587108 (+ 3212us) server_negotiation.cc:407] Connection header received
0620 22:59:35.587289 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:35.587295 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:35.587389 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:35.587499 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:35.588506 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:35.589684 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:35.591554 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:35.592174 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:35.592318 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:35.594424 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:35.594431 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:35.594435 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:35.594470 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:35.597604 (+ 3134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:35.598360 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:35.598365 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:35.598368 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:35.598512 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:35.598973 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:35.598977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:35.598980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:35.599317 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:35.599462 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:35.600051 (+ 589us) server_negotiation.cc:299] Negotiation successful
0620 22:59:35.600266 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260620 22:59:35.600636 9719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:35.583585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.0.0.1:37966)
0620 22:59:35.586972 (+ 3387us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:35.586993 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:35.587150 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:35.587554 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:35.587559 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:35.588062 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:35.588350 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:35.588363 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:35.589833 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:35.589839 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:35.591417 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:35.591431 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:35.592334 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:35.592341 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:35.592513 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:35.593183 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:35.593206 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:35.594210 (+ 1004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:35.597742 (+ 3532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:35.597747 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:35.597752 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:35.598241 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:35.598681 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:35.598686 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:35.598690 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:35.598862 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:35.599532 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:35.599537 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:35.599716 (+ 179us) client_negotiation.cc:769] Sending connection context
0620 22:59:35.599926 (+ 210us) client_negotiation.cc:240] Negotiation successful
0620 22:59:35.600227 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3253,"thread_start_us":3121,"threads_started":1}
I20260620 22:59:35.601902 9169 tablet_copy_service.cc:140] P 988aa53a4d75433ba11c4b68df409c91: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 65dcb1d2184d4d41bb04de52e7e5e2aa ({username='oryx', principal='oryx/127.2.237.61@KRBTEST.COM'} at 127.0.0.1:37966)
I20260620 22:59:35.602207 9169 tablet_copy_service.cc:161] P 988aa53a4d75433ba11c4b68df409c91: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 65dcb1d2184d4d41bb04de52e7e5e2aa at {username='oryx', principal='oryx/127.2.237.61@KRBTEST.COM'} at 127.0.0.1:37966: session id = 65dcb1d2184d4d41bb04de52e7e5e2aa-00000000000000000000000000000000
I20260620 22:59:35.606542 9169 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91: Tablet Copy: opened 0 blocks and 1 log segments
I20260620 22:59:35.609083 9610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260620 22:59:35.616753 9610 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa: tablet copy: Starting download of 0 data blocks...
I20260620 22:59:35.617179 9610 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa: tablet copy: Starting download of 1 WAL segments...
I20260620 22:59:35.620445 9610 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260620 22:59:35.625156 9169 tablet_copy_service.cc:342] P 988aa53a4d75433ba11c4b68df409c91: Request end of tablet copy session 65dcb1d2184d4d41bb04de52e7e5e2aa-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.2.237.61@KRBTEST.COM'} at 127.0.0.1:37966
I20260620 22:59:35.625375 9169 tablet_copy_service.cc:433] P 988aa53a4d75433ba11c4b68df409c91: ending tablet copy session 65dcb1d2184d4d41bb04de52e7e5e2aa-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 65dcb1d2184d4d41bb04de52e7e5e2aa
I20260620 22:59:35.636831 9610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:35.639488 9720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:35.640342 9721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:35.650835 9610 server_base.cc:1034] running on GCE node
W20260620 22:59:35.652233 9724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:35.652776 9610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:35.653466 9610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:35.654731 9610 hybrid_clock.cc:648] HybridClock initialized: now 1781996375654656 us; error 43 us; skew 500 ppm
I20260620 22:59:35.654965 9610 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:35 dist-test-slave-f63b krb5kdc[9104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996375, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:35.657160 9610 init.cc:377] Logged in from keytab as oryx/127.2.237.61@KRBTEST.COM (short username oryx)
I20260620 22:59:35.658403 9610 webserver.cc:460] Webserver started at http://127.2.237.61:45881/ using document root <none> and password file <none>
I20260620 22:59:35.658775 9610 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:35.658857 9610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:35.661866 9610 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260620 22:59:35.663873 9730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:35.664384 9610 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 22:59:35.664525 9610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-1/wal
uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa"
format_stamp: "Formatted at 2026-06-20 22:59:35 on dist-test-slave-f63b"
server_key: "0c9e8e3a9ee82d76ac7a69f1962ba821"
server_key_iv: "d79f8813014c7406356563f3680fd2e5"
server_key_version: "encryptionkey@0"
I20260620 22:59:35.664686 9610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:35.687743 9610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:35.691887 9610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:35.711480 9610 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.61:38995
I20260620 22:59:35.711511 9781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.61:38995 every 8 connection(s)
I20260620 22:59:35.712426 9610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-1/data/info.pb
I20260620 22:59:35.717867 9782 sys_catalog.cc:263] Verifying existing consensus state
I20260620 22:59:35.721678 9782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa: Bootstrap starting.
I20260620 22:59:35.737352 9782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 22:59:35.737788 9782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa: Bootstrap complete.
I20260620 22:59:35.738747 9782 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: NON_VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: true } }
I20260620 22:59:35.738982 9782 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 65dcb1d2184d4d41bb04de52e7e5e2aa, State: Initialized, Role: LEARNER
I20260620 22:59:35.739200 9782 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: NON_VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: true } }
I20260620 22:59:35.740173 9782 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [sys.catalog]: configured and running, proceeding with master startup.
I20260620 22:59:35.740010 9785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: NON_VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: true } } }
I20260620 22:59:35.740523 9785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [sys.catalog]: This master's current role is: LEARNER
I20260620 22:59:35.751165 9796 catalog_manager.cc:1259] Loaded cluster ID: 77a818f56f2d4bb9a22550a45993c572
I20260620 22:59:35.751267 9796 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa: loading cluster ID for follower catalog manager: success
I20260620 22:59:35.754922 9796 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa: acquiring CA information for follower catalog manager: success
I20260620 22:59:35.757521 9796 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 22:59:35.910194 9605 heartbeater.cc:503] Master 127.2.237.62:36895 was elected leader, sending a full tablet report...
W20260620 22:59:35.944260 9711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:35.944546 9711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:35.944653 9711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:35.952139 9711 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 22:59:35.952248 9711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:35.952293 9711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:35.952342 9711 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 22:59:35.952379 9711 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 22:59:35.962184 9711 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:39019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:36895,127.2.237.61:38995,127.2.237.60:42911
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.60.keytab
--principal=oryx/127.2.237.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.60:42911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.237.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:35.963564 9711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:35.964833 9711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:35.976897 9804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:35.977000 9801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:35.976958 9802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:35.977593 9711 server_base.cc:1034] running on GCE node
I20260620 22:59:35.978415 9711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:35.979880 9711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:35.981102 9711 hybrid_clock.cc:648] HybridClock initialized: now 1781996375981041 us; error 62 us; skew 500 ppm
I20260620 22:59:35.981523 9711 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:35 dist-test-slave-f63b krb5kdc[9104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996375, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:35.985044 9711 init.cc:377] Logged in from keytab as oryx/127.2.237.60@KRBTEST.COM (short username oryx)
I20260620 22:59:35.986707 9711 webserver.cc:460] Webserver started at http://127.2.237.60:35903/ using document root <none> and password file <none>
I20260620 22:59:35.987408 9711 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:35.987547 9711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:35.987851 9711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:35.990954 9711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-2/data/instance:
uuid: "c582ab7db2fb468aaa0862be09f6e534"
format_stamp: "Formatted at 2026-06-20 22:59:35 on dist-test-slave-f63b"
server_key: "c6adc66f4be6547d89a700556e117638"
server_key_iv: "dc1104c0f1e274313543101364c1ffa4"
server_key_version: "encryptionkey@0"
I20260620 22:59:35.991727 9711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-2/wal/instance:
uuid: "c582ab7db2fb468aaa0862be09f6e534"
format_stamp: "Formatted at 2026-06-20 22:59:35 on dist-test-slave-f63b"
server_key: "c6adc66f4be6547d89a700556e117638"
server_key_iv: "dc1104c0f1e274313543101364c1ffa4"
server_key_version: "encryptionkey@0"
I20260620 22:59:35.996898 9711 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260620 22:59:36.000303 9811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:36.001709 9711 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 22:59:36.001883 9711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-2/wal
uuid: "c582ab7db2fb468aaa0862be09f6e534"
format_stamp: "Formatted at 2026-06-20 22:59:35 on dist-test-slave-f63b"
server_key: "c6adc66f4be6547d89a700556e117638"
server_key_iv: "dc1104c0f1e274313543101364c1ffa4"
server_key_version: "encryptionkey@0"
I20260620 22:59:36.002125 9711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:36.016877 9711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:36.021174 9711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:36.021456 9711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:36.040743 9711 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.60:42911
I20260620 22:59:36.040764 9862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.60:42911 every 8 connection(s)
I20260620 22:59:36.042234 9711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-2/data/info.pb
I20260620 22:59:36.046373 9863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:36.051553 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 9711
I20260620 22:59:36.053582 9863 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 22:59:36 dist-test-slave-f63b krb5kdc[9104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.237.60@KRBTEST.COM
I20260620 22:59:36.077550 9868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:36.055194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:38995 (local address 127.0.0.1:40528)
0620 22:59:36.070272 (+ 15078us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:36.070293 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:36.070400 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:36.071427 (+ 1027us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:36.071433 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:36.071465 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:59:36.071817 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:36.071835 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.073460 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.073465 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:36.075049 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:36.075063 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.076680 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.076687 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:36.076875 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:36.076939 (+ 64us) client_negotiation.cc:769] Sending connection context
0620 22:59:36.077050 (+ 111us) client_negotiation.cc:240] Negotiation successful
0620 22:59:36.077203 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14970,"mutex_wait_us":31,"thread_start_us":15284,"threads_started":1}
I20260620 22:59:36.077574 9869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:36.055386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40528 (local address 127.2.237.61:38995)
0620 22:59:36.071035 (+ 15649us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:36.071042 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:36.071063 (+ 21us) server_negotiation.cc:407] Connection header received
0620 22:59:36.071132 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:36.071136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:36.071231 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:36.071367 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:59:36.071987 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.073315 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.075211 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.076485 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.076643 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:36.076917 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:36.077087 (+ 170us) server_negotiation.cc:299] Negotiation successful
0620 22:59:36.077242 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15502,"thread_start_us":15391,"threads_started":1}
Jun 20 22:59:36 dist-test-slave-f63b krb5kdc[9104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996375, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.60@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM
I20260620 22:59:36.079229 9757 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260620 22:59:36.079798 9785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 988aa53a4d75433ba11c4b68df409c91. Latest consensus state: current_term: 1 leader_uuid: "988aa53a4d75433ba11c4b68df409c91" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: NON_VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: true } } }
I20260620 22:59:36.080057 9785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [sys.catalog]: This master's current role is: LEARNER
I20260620 22:59:36.084127 9867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:36.053664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40304 (local address 127.2.237.60:42911)
0620 22:59:36.054609 (+ 945us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:36.054628 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:36.054678 (+ 50us) server_negotiation.cc:407] Connection header received
0620 22:59:36.057827 (+ 3149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:36.057865 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:36.058469 (+ 604us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:36.061248 (+ 2779us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:36.061893 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.064845 (+ 2952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.067289 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.068741 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.071033 (+ 2292us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:36.074761 (+ 3728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:36.074770 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:36.074785 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:36.074831 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:36.079009 (+ 4178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:36.079860 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:36.079869 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:36.079879 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:36.080055 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:36.080514 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:36.080519 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:36.080523 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:36.081265 (+ 742us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:36.081555 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:36.082090 (+ 535us) server_negotiation.cc:299] Negotiation successful
0620 22:59:36.082750 (+ 660us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":415,"threads_started":1}
I20260620 22:59:36.086361 9340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:36.057705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37970 (local address 127.2.237.62:36895)
0620 22:59:36.057907 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:36.057915 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:36.068454 (+ 10539us) server_negotiation.cc:407] Connection header received
0620 22:59:36.069657 (+ 1203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:36.069667 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:36.069771 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:36.069855 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:36.071558 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.072812 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.074840 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.075379 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.075499 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:36.080225 (+ 4726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:36.080235 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:36.080240 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:36.080280 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:36.083663 (+ 3383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:36.084415 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:36.084419 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:36.084421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:36.084580 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:36.084957 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:36.084966 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:36.084968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:36.085208 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:36.085351 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:36.085771 (+ 420us) server_negotiation.cc:299] Negotiation successful
0620 22:59:36.085985 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":62}
I20260620 22:59:36.086414 9870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:36.057083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.0.0.1:37970)
0620 22:59:36.068279 (+ 11196us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:36.068343 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:36.069447 (+ 1104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:36.070135 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:36.070142 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:36.071016 (+ 874us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:36.071371 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:36.071383 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.072991 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.072998 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:36.074714 (+ 1716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:36.074726 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.075551 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.075560 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:36.075800 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:36.076449 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:36.076465 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:36.080014 (+ 3549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:36.083783 (+ 3769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:36.083791 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:36.083796 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:36.084237 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:36.084690 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:36.084696 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:36.084700 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:36.084850 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:36.085325 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:36.085332 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:36.085496 (+ 164us) client_negotiation.cc:769] Sending connection context
0620 22:59:36.085673 (+ 177us) client_negotiation.cc:240] Negotiation successful
0620 22:59:36.086002 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10853,"spinlock_wait_cycles":23424,"thread_start_us":11396,"threads_started":1}
I20260620 22:59:36.087647 9863 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 22:59:36 dist-test-slave-f63b krb5kdc[9104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996375, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.60@KRBTEST.COM for oryx/127.2.237.61@KRBTEST.COM
I20260620 22:59:36.098889 9869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:36.088399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40536 (local address 127.2.237.61:38995)
0620 22:59:36.088540 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:36.088544 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:36.088559 (+ 15us) server_negotiation.cc:407] Connection header received
0620 22:59:36.088669 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:36.088673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:36.088731 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:36.088808 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:36.089362 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.090038 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.091214 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.091596 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.091687 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:36.094294 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:36.094298 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:36.094301 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:36.094328 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:36.096725 (+ 2397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:36.097368 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:36.097372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:36.097374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:36.097487 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:36.097816 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:36.097818 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:36.097820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:36.098046 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:36.098146 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:36.098442 (+ 296us) server_negotiation.cc:299] Negotiation successful
0620 22:59:36.098674 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260620 22:59:36.098929 9870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:36.088302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:38995 (local address 127.0.0.1:40536)
0620 22:59:36.088480 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:36.088511 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:36.088583 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:36.088833 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:36.088836 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:36.089123 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:36.089246 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:36.089253 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.090139 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.090142 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:36.091099 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:36.091109 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.091715 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.091719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:36.091826 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:36.092346 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:36.092361 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:36.094148 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:36.096844 (+ 2696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:36.096848 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:36.096851 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:36.097259 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:36.097599 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:36.097602 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:36.097605 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:36.097730 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:36.098136 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:36.098139 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:36.098261 (+ 122us) client_negotiation.cc:769] Sending connection context
0620 22:59:36.098372 (+ 111us) client_negotiation.cc:240] Negotiation successful
0620 22:59:36.098673 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260620 22:59:36.099941 9863 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.60" port: 42911 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 22:59:36 dist-test-slave-f63b krb5kdc[9104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996375, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.60@KRBTEST.COM for oryx/127.2.237.60@KRBTEST.COM
I20260620 22:59:36.112356 9870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:36.100729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:42911 (local address 127.0.0.1:40310)
0620 22:59:36.100867 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:36.100882 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:36.100984 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:36.101292 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:36.101297 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:36.101625 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:36.101747 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:36.101754 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.102767 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.102776 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:36.103572 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:36.103582 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.104261 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.104267 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:36.104445 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:36.104938 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:36.104953 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:36.107114 (+ 2161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:36.110032 (+ 2918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:36.110039 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:36.110044 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:36.110372 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:36.110752 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:36.110756 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:36.110760 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:36.110897 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:36.111396 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:36.111400 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:36.111534 (+ 134us) client_negotiation.cc:769] Sending connection context
0620 22:59:36.111681 (+ 147us) client_negotiation.cc:240] Negotiation successful
0620 22:59:36.111935 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260620 22:59:36.112380 9867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:36.100894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40310 (local address 127.2.237.60:42911)
0620 22:59:36.101026 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:36.101033 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:36.101050 (+ 17us) server_negotiation.cc:407] Connection header received
0620 22:59:36.101093 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:36.101096 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:36.101164 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:36.101279 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:36.101867 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.102613 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.103708 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.104106 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.104233 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:36.107317 (+ 3084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:36.107321 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:36.107325 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:36.107350 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:36.109908 (+ 2558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:36.110479 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:36.110486 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:36.110490 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:36.110649 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:36.110994 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:36.110998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:36.111001 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:36.111277 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:36.111420 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:36.111748 (+ 328us) server_negotiation.cc:299] Negotiation successful
0620 22:59:36.112068 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":128,"server-negotiator.queue_time_us":44}
I20260620 22:59:36.117713 9863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534: Bootstrap starting.
I20260620 22:59:36.120807 9863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:36.121765 9863 log.cc:824] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534: Log is configured to *not* fsync() on all Append() calls
I20260620 22:59:36.124543 9863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534: No bootstrap required, opened a new log
I20260620 22:59:36.131341 9863 raft_consensus.cc:348] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 42911 } }
I20260620 22:59:36.131628 9863 raft_consensus.cc:374] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:36.131762 9863 raft_consensus.cc:729] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c582ab7db2fb468aaa0862be09f6e534, State: Initialized, Role: FOLLOWER
I20260620 22:59:36.132308 9863 consensus_queue.cc:260] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 42911 } }
I20260620 22:59:36.133157 9872 sys_catalog.cc:455] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 42911 } } }
I20260620 22:59:36.133517 9872 sys_catalog.cc:458] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 22:59:36.134533 9863 sys_catalog.cc:564] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [sys.catalog]: configured and running, proceeding with master startup.
W20260620 22:59:36.144502 9883 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 22:59:36.144692 9883 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 22:59:36.147423 9711 master_runner.cc:428] Detected that this master c582ab7db2fb468aaa0862be09f6e534 is joining an existing cluster
I20260620 22:59:36.147583 9711 master_runner.cc:432] Initiating AddMaster RPC to add 127.2.237.60:42911
I20260620 22:59:36.168749 9869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:36.152336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40550 (local address 127.2.237.61:38995)
0620 22:59:36.152545 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:36.152551 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:36.153311 (+ 760us) server_negotiation.cc:407] Connection header received
0620 22:59:36.153564 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:36.153569 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:36.153644 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:36.153709 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:36.155212 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.156235 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.158149 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.158811 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.158898 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:36.161333 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:36.161339 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:36.161344 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:36.161386 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:36.165664 (+ 4278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:36.166379 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:36.166384 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:36.166388 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:36.166538 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:36.167025 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:36.167031 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:36.167034 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:36.167380 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:36.167503 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:36.168122 (+ 619us) server_negotiation.cc:299] Negotiation successful
0620 22:59:36.168370 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260620 22:59:36.169477 9340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:36.151874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37978 (local address 127.2.237.62:36895)
0620 22:59:36.152495 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:36.152501 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:36.152518 (+ 17us) server_negotiation.cc:407] Connection header received
0620 22:59:36.152687 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:36.152692 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:36.152782 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:36.152880 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:36.154539 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.155694 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.157773 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.158416 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.158518 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:36.161332 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:36.161339 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:36.161344 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:36.161386 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:36.165655 (+ 4269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:36.166921 (+ 1266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:36.166929 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:36.166935 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:36.167097 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:36.167577 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:36.167582 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:36.167585 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:36.167916 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:36.168042 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:36.168612 (+ 570us) server_negotiation.cc:299] Negotiation successful
0620 22:59:36.169133 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512}
I20260620 22:59:36.169847 9890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:36.151430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:38995 (local address 127.0.0.1:40550)
0620 22:59:36.153224 (+ 1794us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:36.153243 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:36.153407 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:36.153903 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:36.153908 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:36.154742 (+ 834us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:36.155042 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:36.155058 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.156423 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.156429 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:36.158007 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:36.158021 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.159180 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.159186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:36.159378 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:36.160013 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:36.160033 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:36.161138 (+ 1105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:36.165803 (+ 4665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:36.165808 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:36.165812 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:36.166236 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:36.166711 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:36.166715 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:36.166719 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:36.166900 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:36.167587 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:36.167592 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:36.167817 (+ 225us) client_negotiation.cc:769] Sending connection context
0620 22:59:36.168997 (+ 1180us) client_negotiation.cc:240] Negotiation successful
0620 22:59:36.169325 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1664,"thread_start_us":1456,"threads_started":1}
I20260620 22:59:36.171062 9889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:36.151229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.0.0.1:37978)
0620 22:59:36.152355 (+ 1126us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:36.152375 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:36.152523 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:36.153120 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:36.153124 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:36.153827 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:36.154333 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:36.154349 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.155854 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.155859 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:36.157612 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:36.157625 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.158812 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.158819 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:36.158977 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:36.159767 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:36.159802 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:36.161138 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:36.166201 (+ 5063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:36.166205 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:36.166209 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:36.166789 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:36.167257 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:36.167261 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:36.167265 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:36.167442 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:36.168129 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:36.168134 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:36.168341 (+ 207us) client_negotiation.cc:769] Sending connection context
0620 22:59:36.170168 (+ 1827us) client_negotiation.cc:240] Negotiation successful
0620 22:59:36.170487 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"spinlock_wait_cycles":36736,"thread_start_us":1281,"threads_started":1}
I20260620 22:59:36.181308 9867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:36.151704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40326 (local address 127.2.237.60:42911)
0620 22:59:36.152354 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:36.152360 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:36.165257 (+ 12897us) server_negotiation.cc:407] Connection header received
0620 22:59:36.165584 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:36.165589 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:36.165660 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:36.165727 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:36.167459 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.168760 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.171085 (+ 2325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.171915 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.172018 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:36.175209 (+ 3191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:36.175216 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:36.175220 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:36.175251 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:36.178694 (+ 3443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:36.179277 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:36.179282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:36.179286 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:36.179399 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:36.179751 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:36.179756 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:36.179759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:36.180062 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:36.180225 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:36.180615 (+ 390us) server_negotiation.cc:299] Negotiation successful
0620 22:59:36.180943 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":139,"server-negotiator.queue_time_us":220}
I20260620 22:59:36.181330 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:36.152546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:42911 (local address 127.0.0.1:40326)
0620 22:59:36.165153 (+ 12607us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:36.165177 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:36.165389 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:36.165960 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:36.165965 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:36.166905 (+ 940us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:36.167280 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:36.167293 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.168990 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.168996 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:36.170866 (+ 1870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:36.170888 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.172345 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.172353 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:36.173002 (+ 649us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:36.173706 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:36.173729 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:36.174983 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:36.178822 (+ 3839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:36.178827 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:36.178831 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:36.179161 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:36.179505 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:36.179510 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:36.179515 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:36.179632 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:36.180170 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:36.180174 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:36.180335 (+ 161us) client_negotiation.cc:769] Sending connection context
0620 22:59:36.180474 (+ 139us) client_negotiation.cc:240] Negotiation successful
0620 22:59:36.180816 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12463,"thread_start_us":12331,"threads_started":1}
Jun 20 22:59:36 dist-test-slave-f63b krb5kdc[9104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996373, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.62@KRBTEST.COM for oryx/127.2.237.60@KRBTEST.COM
I20260620 22:59:36.190112 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 9210
I20260620 22:59:36.194747 9892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:36.176977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40336 (local address 127.2.237.60:42911)
0620 22:59:36.177882 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:36.177888 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:36.177915 (+ 27us) server_negotiation.cc:407] Connection header received
0620 22:59:36.178008 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:36.178014 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:36.178114 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:36.179215 (+ 1101us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:36.179584 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.181155 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.183816 (+ 2661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.184596 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.184727 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:36.188117 (+ 3390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:36.188122 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:36.188125 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:36.188151 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:36.191509 (+ 3358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:36.192333 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:36.192341 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:36.192344 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:36.192505 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:36.192924 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:36.192927 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:36.192929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:36.193172 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:36.193326 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:36.194106 (+ 780us) server_negotiation.cc:299] Negotiation successful
0620 22:59:36.194308 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":762,"thread_start_us":574,"threads_started":1}
I20260620 22:59:36.195176 9868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:36.176891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:42911 (local address 127.0.0.1:40336)
0620 22:59:36.177187 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:36.177207 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:36.177318 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:36.178335 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:36.178343 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:36.178853 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:36.179054 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:36.179067 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.181343 (+ 2276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.181350 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:36.183467 (+ 2117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:36.183483 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.184727 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.184731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:36.184841 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:36.185354 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:36.185370 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:36.187898 (+ 2528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:36.191678 (+ 3780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:36.191686 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:36.191690 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:36.192166 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:36.192635 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:36.192642 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:36.192649 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:36.192811 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:36.193297 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:36.193302 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:36.193865 (+ 563us) client_negotiation.cc:769] Sending connection context
0620 22:59:36.193996 (+ 131us) client_negotiation.cc:240] Negotiation successful
0620 22:59:36.194596 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175}
I20260620 22:59:36.196499 9149 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.2.237.60:42911: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534"
member_type: NON_VOTER
last_known_addr {
host: "127.2.237.60"
port: 42911
}
attrs {
promote: true
}
}
dest_uuid: "988aa53a4d75433ba11c4b68df409c91"
cas_config_opid_index: 5
I20260620 22:59:36.197530 9149 consensus_queue.cc:237] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: NON_VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: true } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: NON_VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: true } }
I20260620 22:59:36.199291 9866 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER c582ab7db2fb468aaa0862be09f6e534 (127.2.237.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: NON_VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: true } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: NON_VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: true } } }
I20260620 22:59:36.199975 9757 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 1 LEARNER]: Refusing update from remote peer 988aa53a4d75433ba11c4b68df409c91: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260620 22:59:36.200412 9838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:36.201953 9866 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.2.237.60:42911
I20260620 22:59:36.202275 9894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "988aa53a4d75433ba11c4b68df409c91" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: NON_VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: true } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: NON_VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: true } } }
I20260620 22:59:36.202793 9894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:36.203298 9866 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: NON_VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
Jun 20 22:59:36 dist-test-slave-f63b krb5kdc[9104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.237.1@KRBTEST.COM
I20260620 22:59:36.207680 9757 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER c582ab7db2fb468aaa0862be09f6e534 (127.2.237.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: NON_VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: true } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: NON_VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: true } } }
I20260620 22:59:36.212280 9785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "988aa53a4d75433ba11c4b68df409c91" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: NON_VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: true } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: NON_VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: true } } }
I20260620 22:59:36.212561 9785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [sys.catalog]: This master's current role is: LEARNER
I20260620 22:59:36.213487 9866 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91: attempting to promote NON_VOTER 65dcb1d2184d4d41bb04de52e7e5e2aa to VOTER
I20260620 22:59:36.214257 9866 consensus_queue.cc:237] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: NON_VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: true } }
I20260620 22:59:36.216478 9757 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 1 LEARNER]: Refusing update from remote peer 988aa53a4d75433ba11c4b68df409c91: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260620 22:59:36.217078 9894 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260620 22:59:36.217738 9893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:36.191271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59110 (local address 127.2.237.1:41217)
0620 22:59:36.191922 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:36.191934 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:36.191963 (+ 29us) server_negotiation.cc:407] Connection header received
0620 22:59:36.192032 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:36.192042 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:36.192344 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:36.192512 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:36.194516 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.196329 (+ 1813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.198030 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.198969 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.199100 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:36.209309 (+ 10209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:36.209325 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:36.209345 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:36.209390 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:36.213683 (+ 4293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:36.214669 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:36.214679 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:36.214682 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:36.214846 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:36.215282 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:36.215287 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:36.215291 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:36.215728 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:36.216013 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:36.216930 (+ 917us) server_negotiation.cc:299] Negotiation successful
0620 22:59:36.217204 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":539,"threads_started":1}
I20260620 22:59:36.219825 9333 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:36.222946 9711 master.cc:537] Master@127.2.237.60:42911 shutting down...
I20260620 22:59:36.223239 9838 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [term 1 FOLLOWER]: Refusing update from remote peer 988aa53a4d75433ba11c4b68df409c91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260620 22:59:36.224009 9837 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [term 1 FOLLOWER]: Refusing update from remote peer 988aa53a4d75433ba11c4b68df409c91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260620 22:59:36.223726 9756 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 65dcb1d2184d4d41bb04de52e7e5e2aa (127.2.237.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: NON_VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: true } } }
I20260620 22:59:36.226920 9866 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 65dcb1d2184d4d41bb04de52e7e5e2aa (127.2.237.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: NON_VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: true } } }
I20260620 22:59:36.227522 9894 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: NON_VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:59:36.228236 9785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "988aa53a4d75433ba11c4b68df409c91" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: NON_VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: true } } }
I20260620 22:59:36.228494 9785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [sys.catalog]: This master's current role is: FOLLOWER
I20260620 22:59:36.234818 9866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "988aa53a4d75433ba11c4b68df409c91" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: NON_VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: true } } }
I20260620 22:59:36.235392 9866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:36.235128 9897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "988aa53a4d75433ba11c4b68df409c91" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: NON_VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: true } } }
I20260620 22:59:36.236056 9897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:36.242252 9711 raft_consensus.cc:2229] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 22:59:36.242733 9711 raft_consensus.cc:2258] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:36.242944 9711 tablet_replica.cc:330] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534: stopping tablet replica
W20260620 22:59:36.247149 9711 connection.cc:441] server connection from 127.0.0.1:40336 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40336 (request call id 4) could send its response
W20260620 22:59:36.247637 9133 consensus_peers.cc:487] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 -> Peer c582ab7db2fb468aaa0862be09f6e534 (127.2.237.60:42911): Couldn't send request to peer c582ab7db2fb468aaa0862be09f6e534. Status: Network error: recv got EOF from 127.2.237.60:42911 (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260620 22:59:36.249814 9711 master.cc:552] Master@127.2.237.60:42911 shutdown complete.
I20260620 22:59:36.249905 9711 master_runner.cc:305] Clearing existing system tablet
I20260620 22:59:36.250241 9711 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:36.250351 9711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:36.252579 9711 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260620 22:59:36.253984 9901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:36.254273 9711 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:36.254557 9711 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260620 22:59:36.254662 9711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-2/wal
uuid: "c582ab7db2fb468aaa0862be09f6e534"
format_stamp: "Formatted at 2026-06-20 22:59:35 on dist-test-slave-f63b"
server_key: "c6adc66f4be6547d89a700556e117638"
server_key_iv: "dc1104c0f1e274313543101364c1ffa4"
server_key_version: "encryptionkey@0"
I20260620 22:59:36.257932 9711 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:36.265470 9711 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260620 22:59:36.265725 9711 log.cc:1196] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260620 22:59:36.266270 9711 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534: Deleting consensus metadata
I20260620 22:59:36.266562 9711 master_runner.cc:315] Copying system tablet from 127.2.237.62:36895
I20260620 22:59:36.287861 9711 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534: tablet copy: Beginning tablet copy session from remote peer at address 127.2.237.62:36895
I20260620 22:59:36.302282 9340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:36.288665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37986 (local address 127.2.237.62:36895)
0620 22:59:36.288911 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:36.288920 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:36.291102 (+ 2182us) server_negotiation.cc:407] Connection header received
0620 22:59:36.291321 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:36.291327 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:36.291401 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:36.291506 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:36.292332 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.293098 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.294322 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.294786 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.294922 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:36.296706 (+ 1784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:36.296710 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:36.296714 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:36.296743 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:36.299732 (+ 2989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:36.300387 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:36.300395 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:36.300398 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:36.300518 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:36.300886 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:36.300892 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:36.300896 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:36.301158 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:36.301305 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:36.301722 (+ 417us) server_negotiation.cc:299] Negotiation successful
0620 22:59:36.301922 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260620 22:59:36.302347 9907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:36.288657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.0.0.1:37986)
0620 22:59:36.290978 (+ 2321us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:36.291005 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:36.291178 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:36.291532 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:36.291536 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:36.291965 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:36.292151 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:36.292159 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.293254 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.293259 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:36.294175 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:36.294186 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.294943 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.294949 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:36.295146 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:36.295768 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:36.295785 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:36.296503 (+ 718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:36.299870 (+ 3367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:36.299875 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:36.299879 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:36.300270 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:36.300624 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:36.300630 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:36.300634 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:36.300784 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:36.301270 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:36.301275 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:36.301434 (+ 159us) client_negotiation.cc:769] Sending connection context
0620 22:59:36.301574 (+ 140us) client_negotiation.cc:240] Negotiation successful
0620 22:59:36.301887 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2194,"thread_start_us":2079,"threads_started":1}
I20260620 22:59:36.303313 9169 tablet_copy_service.cc:140] P 988aa53a4d75433ba11c4b68df409c91: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer c582ab7db2fb468aaa0862be09f6e534 ({username='oryx', principal='oryx/127.2.237.60@KRBTEST.COM'} at 127.0.0.1:37986)
I20260620 22:59:36.303478 9169 tablet_copy_service.cc:161] P 988aa53a4d75433ba11c4b68df409c91: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer c582ab7db2fb468aaa0862be09f6e534 at {username='oryx', principal='oryx/127.2.237.60@KRBTEST.COM'} at 127.0.0.1:37986: session id = c582ab7db2fb468aaa0862be09f6e534-00000000000000000000000000000000
I20260620 22:59:36.305724 9169 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91: Tablet Copy: opened 0 blocks and 1 log segments
I20260620 22:59:36.307412 9711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260620 22:59:36.312084 9711 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534: tablet copy: Starting download of 0 data blocks...
I20260620 22:59:36.312467 9711 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534: tablet copy: Starting download of 1 WAL segments...
I20260620 22:59:36.314841 9711 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260620 22:59:36.318718 9169 tablet_copy_service.cc:342] P 988aa53a4d75433ba11c4b68df409c91: Request end of tablet copy session c582ab7db2fb468aaa0862be09f6e534-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.2.237.60@KRBTEST.COM'} at 127.0.0.1:37986
I20260620 22:59:36.318858 9169 tablet_copy_service.cc:433] P 988aa53a4d75433ba11c4b68df409c91: ending tablet copy session c582ab7db2fb468aaa0862be09f6e534-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer c582ab7db2fb468aaa0862be09f6e534
I20260620 22:59:36.328613 9711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:36.331224 9908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:36.331888 9909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:36.337002 9711 server_base.cc:1034] running on GCE node
W20260620 22:59:36.338825 9911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:36.339236 9711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:36.339893 9711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:36.341079 9711 hybrid_clock.cc:648] HybridClock initialized: now 1781996376341017 us; error 39 us; skew 500 ppm
I20260620 22:59:36.341318 9711 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:36 dist-test-slave-f63b krb5kdc[9104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996376, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:36.343361 9711 init.cc:377] Logged in from keytab as oryx/127.2.237.60@KRBTEST.COM (short username oryx)
I20260620 22:59:36.347530 9711 webserver.cc:460] Webserver started at http://127.2.237.60:33445/ using document root <none> and password file <none>
I20260620 22:59:36.347877 9711 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:36.347959 9711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:36.350334 9711 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260620 22:59:36.352507 9918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:36.352897 9711 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260620 22:59:36.353019 9711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-2/wal
uuid: "c582ab7db2fb468aaa0862be09f6e534"
format_stamp: "Formatted at 2026-06-20 22:59:35 on dist-test-slave-f63b"
server_key: "c6adc66f4be6547d89a700556e117638"
server_key_iv: "dc1104c0f1e274313543101364c1ffa4"
server_key_version: "encryptionkey@0"
I20260620 22:59:36.353195 9711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:36.363426 9711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:36.367339 9711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:36.379314 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 9210
I20260620 22:59:36.407142 9711 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.60:42911
I20260620 22:59:36.407226 9969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.60:42911 every 8 connection(s)
I20260620 22:59:36.408133 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:41217
--local_ip_for_outbound_sockets=127.2.237.1
--tserver_master_addrs=127.2.237.62:36895
--webserver_port=35785
--webserver_interface=127.2.237.1
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.1.keytab
--principal=oryx/127.2.237.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:39019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.2.237.62:36895,127.2.237.61:38995
--tserver_master_addrs=127.2.237.62:36895,127.2.237.61:38995,127.2.237.60:42911 with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 22:59:36.408201 9711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/master-2/data/info.pb
I20260620 22:59:36.415365 9970 sys_catalog.cc:263] Verifying existing consensus state
I20260620 22:59:36.419483 9970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534: Bootstrap starting.
I20260620 22:59:36.438962 9970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 22:59:36.439527 9970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534: Bootstrap complete.
I20260620 22:59:36.440551 9970 raft_consensus.cc:348] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: NON_VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: true } }
I20260620 22:59:36.440857 9970 raft_consensus.cc:729] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c582ab7db2fb468aaa0862be09f6e534, State: Initialized, Role: LEARNER
I20260620 22:59:36.441103 9970 consensus_queue.cc:260] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: NON_VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: true } }
I20260620 22:59:36.442114 9970 sys_catalog.cc:564] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 22:59:36.441846 9974 sys_catalog.cc:455] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: NON_VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: true } } }
I20260620 22:59:36.442385 9974 sys_catalog.cc:458] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [sys.catalog]: This master's current role is: LEARNER
I20260620 22:59:36.454550 9985 catalog_manager.cc:1259] Loaded cluster ID: 77a818f56f2d4bb9a22550a45993c572
I20260620 22:59:36.454671 9985 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534: loading cluster ID for follower catalog manager: success
I20260620 22:59:36.458016 9985 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534: acquiring CA information for follower catalog manager: success
I20260620 22:59:36.460355 9985 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 22:59:36.638675 9868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:36.631645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:42911 (local address 127.0.0.1:42740)
0620 22:59:36.631898 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:36.631919 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:36.632094 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:36.632989 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:36.632995 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:36.633021 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:59:36.633255 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:36.633269 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.635188 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.635195 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:36.636443 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:36.636455 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.638005 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.638010 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:36.638156 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:36.638189 (+ 33us) client_negotiation.cc:769] Sending connection context
0620 22:59:36.638273 (+ 84us) client_negotiation.cc:240] Negotiation successful
0620 22:59:36.638373 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137}
I20260620 22:59:36.638793 9986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:36.631837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42740 (local address 127.2.237.60:42911)
0620 22:59:36.632619 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:36.632625 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:36.632644 (+ 19us) server_negotiation.cc:407] Connection header received
0620 22:59:36.632714 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:36.632719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:36.632803 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:36.632937 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:59:36.633418 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.634991 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.636611 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.637842 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.637967 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:36.638212 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:36.638311 (+ 99us) server_negotiation.cc:299] Negotiation successful
0620 22:59:36.638434 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":476,"threads_started":1}
I20260620 22:59:36.639886 9945 raft_consensus.cc:1203] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.7] Dedup: 1.7->[]
I20260620 22:59:36.640458 9974 sys_catalog.cc:455] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 988aa53a4d75433ba11c4b68df409c91. Latest consensus state: current_term: 1 leader_uuid: "988aa53a4d75433ba11c4b68df409c91" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: NON_VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: true } } }
I20260620 22:59:36.640753 9974 sys_catalog.cc:458] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [sys.catalog]: This master's current role is: LEARNER
W20260620 22:59:36.760737 9972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:36.761018 9972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:36.761161 9972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:36.768694 9972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:36.768803 9972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:36.768980 9972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 22:59:36.779091 9972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:39019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.1.keytab
--principal=oryx/127.2.237.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:41217
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=35785
--tserver_master_addrs=127.2.237.62:36895,127.2.237.61:38995,127.2.237.60:42911
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:36.780426 9972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:36.781716 9972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:36.795173 9994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:36.795188 9992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:36.795188 9991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:36.795894 9972 server_base.cc:1034] running on GCE node
I20260620 22:59:36.796569 9972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:36.797506 9972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:36.798722 9972 hybrid_clock.cc:648] HybridClock initialized: now 1781996376798675 us; error 53 us; skew 500 ppm
I20260620 22:59:36.799042 9972 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:36 dist-test-slave-f63b krb5kdc[9104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996376, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:36.802277 9972 init.cc:377] Logged in from keytab as oryx/127.2.237.1@KRBTEST.COM (short username oryx)
I20260620 22:59:36.803712 9972 webserver.cc:460] Webserver started at http://127.2.237.1:35785/ using document root <none> and password file <none>
I20260620 22:59:36.804351 9972 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:36.804458 9972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:36.809986 9972 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260620 22:59:36.812559 10001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:36.813813 9972 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 22:59:36.813980 9972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "8154a02194f741cbb6596035cec6fcf0"
format_stamp: "Formatted at 2026-06-20 22:59:33 on dist-test-slave-f63b"
server_key: "abc5e1d34256f74cdfb5abcdb9124685"
server_key_iv: "e6f7edc3087bfc7db5d629c515c5abd1"
server_key_version: "encryptionkey@0"
I20260620 22:59:36.814985 9972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:36.842077 9972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:36.846534 9972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:36.846840 9972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:36.847875 9972 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:59:36.849406 9972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:36.849521 9972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:36.849647 9972 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:36.849731 9972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:36.885313 9972 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:41217
I20260620 22:59:36.885426 10113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:41217 every 8 connection(s)
I20260620 22:59:36.886816 9972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 22:59:36.888000 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 9972
I20260620 22:59:36.888409 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 9344
Jun 20 22:59:36 dist-test-slave-f63b krb5kdc[9104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996376, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.61@KRBTEST.COM
Jun 20 22:59:36 dist-test-slave-f63b krb5kdc[9104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996376, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.60@KRBTEST.COM
Jun 20 22:59:36 dist-test-slave-f63b krb5kdc[9104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996376, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM
Jun 20 22:59:36 dist-test-slave-f63b krb5kdc[9104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.237.2@KRBTEST.COM
I20260620 22:59:36.927961 9986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:36.896153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:38519 (local address 127.2.237.60:42911)
0620 22:59:36.896410 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:36.896420 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:36.903081 (+ 6661us) server_negotiation.cc:407] Connection header received
0620 22:59:36.906349 (+ 3268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:36.906359 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:36.906471 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:36.906563 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:36.909718 (+ 3155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.910869 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.914261 (+ 3392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.914932 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.915024 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:36.920491 (+ 5467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:36.920497 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:36.920502 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:36.920540 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:36.924110 (+ 3570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:36.925130 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:36.925137 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:36.925143 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:36.925295 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:36.925825 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:36.925830 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:36.925834 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:36.926208 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:36.926328 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:36.927356 (+ 1028us) server_negotiation.cc:299] Negotiation successful
0620 22:59:36.927619 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260620 22:59:36.928205 10120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:36.889726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:45697 (local address 127.2.237.62:36895)
0620 22:59:36.901534 (+ 11808us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:36.901538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:36.902814 (+ 1276us) server_negotiation.cc:407] Connection header received
0620 22:59:36.906355 (+ 3541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:36.906365 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:36.906479 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:36.906630 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:36.909720 (+ 3090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.910957 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.913817 (+ 2860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.914422 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.914517 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:36.921030 (+ 6513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:36.921040 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:36.921047 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:36.921089 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:36.925232 (+ 4143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:36.926057 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:36.926062 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:36.926066 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:36.926194 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:36.926648 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:36.926653 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:36.926656 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:36.927019 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:36.927134 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:36.927627 (+ 493us) server_negotiation.cc:299] Negotiation successful
0620 22:59:36.927851 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11700,"thread_start_us":11601,"threads_started":1}
I20260620 22:59:36.929286 10122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:36.890603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.2.237.1:45697)
0620 22:59:36.902561 (+ 11958us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:36.902671 (+ 110us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:36.905967 (+ 3296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:36.907236 (+ 1269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:36.907246 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:36.908324 (+ 1078us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:36.909482 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:36.909521 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.911159 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.911164 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:36.913634 (+ 2470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:36.913648 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.914927 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.914933 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:36.915329 (+ 396us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:36.916280 (+ 951us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:36.916314 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:36.920843 (+ 4529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:36.925378 (+ 4535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:36.925382 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:36.925386 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:36.925931 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:36.926341 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:36.926348 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:36.926351 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:36.926502 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:36.927223 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:36.927230 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:36.927357 (+ 127us) client_negotiation.cc:769] Sending connection context
0620 22:59:36.928459 (+ 1102us) client_negotiation.cc:240] Negotiation successful
0620 22:59:36.928765 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11676,"mutex_wait_us":357,"spinlock_wait_cycles":69248,"thread_start_us":12470,"threads_started":1}
I20260620 22:59:36.929919 10124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:36.896296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:52233 (local address 127.2.237.61:38995)
0620 22:59:36.906195 (+ 9899us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:36.906202 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:36.907330 (+ 1128us) server_negotiation.cc:407] Connection header received
0620 22:59:36.907397 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:36.907404 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:36.907496 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:36.907600 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:36.909780 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.910869 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.913901 (+ 3032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.914559 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.914682 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:36.920374 (+ 5692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:36.920382 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:36.920387 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:36.920431 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:36.924487 (+ 4056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:36.925492 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:36.925499 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:36.925504 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:36.925661 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:36.926123 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:36.926129 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:36.926133 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:36.926512 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:36.926692 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:36.929324 (+ 2632us) server_negotiation.cc:299] Negotiation successful
0620 22:59:36.929548 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9777,"thread_start_us":9684,"threads_started":1}
I20260620 22:59:36.929940 10121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:36.895574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:42911 (local address 127.2.237.1:38519)
0620 22:59:36.902987 (+ 7413us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:36.903003 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:36.905963 (+ 2960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:36.907233 (+ 1270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:36.907245 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:36.908325 (+ 1080us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:36.909486 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:36.909519 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.911051 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.911056 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:36.914104 (+ 3048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:36.914115 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.915303 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.915308 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:36.915528 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:36.916359 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:36.916393 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:36.920253 (+ 3860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:36.924346 (+ 4093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:36.924355 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:36.924373 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:36.924989 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:36.925460 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:36.925464 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:36.925467 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:36.925693 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:36.926417 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:36.926425 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:36.926916 (+ 491us) client_negotiation.cc:769] Sending connection context
0620 22:59:36.928897 (+ 1981us) client_negotiation.cc:240] Negotiation successful
0620 22:59:36.929174 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7302,"mutex_wait_us":471,"spinlock_wait_cycles":50432,"thread_start_us":6525,"threads_started":1}
I20260620 22:59:36.930591 10123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:36.895867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:38995 (local address 127.2.237.1:52233)
0620 22:59:36.907097 (+ 11230us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:36.907118 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:36.907233 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:36.908014 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:36.908018 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:36.908603 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:36.909613 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:36.909624 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.911051 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.911056 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:36.913744 (+ 2688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:36.913758 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.915004 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.915010 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:36.915297 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:36.916467 (+ 1170us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:36.916492 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:36.920067 (+ 3575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:36.924689 (+ 4622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:36.924695 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:36.924699 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:36.925344 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:36.925794 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:36.925799 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:36.925803 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:36.925982 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:36.926767 (+ 785us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:36.926775 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:36.926919 (+ 144us) client_negotiation.cc:769] Sending connection context
0620 22:59:36.927232 (+ 313us) client_negotiation.cc:240] Negotiation successful
0620 22:59:36.927776 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11119,"mutex_wait_us":499,"spinlock_wait_cycles":11648,"thread_start_us":10941,"threads_started":1}
I20260620 22:59:36.931141 10114 heartbeater.cc:347] Connected to a master server at 127.2.237.62:36895
I20260620 22:59:36.931270 10115 heartbeater.cc:347] Connected to a master server at 127.2.237.60:42911
I20260620 22:59:36.931469 10114 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:36.931469 10115 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:36.931944 10119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:36.889515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33906 (local address 127.2.237.2:33799)
0620 22:59:36.907448 (+ 17933us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:36.907458 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:36.907492 (+ 34us) server_negotiation.cc:407] Connection header received
0620 22:59:36.907582 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:36.907592 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:36.907950 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:36.909214 (+ 1264us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:36.910217 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.911753 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.915775 (+ 4022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:36.916692 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:36.916866 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:36.924123 (+ 7257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:36.924151 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:36.924171 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:36.924219 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:36.928038 (+ 3819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:36.928927 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:36.928935 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:36.928938 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:36.929092 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:36.929541 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:36.929545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:36.929548 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:36.929919 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:36.931099 (+ 1180us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:36.931268 (+ 169us) server_negotiation.cc:299] Negotiation successful
0620 22:59:36.931520 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17752,"thread_start_us":18234,"threads_started":1}
I20260620 22:59:36.932431 10115 heartbeater.cc:511] Master 127.2.237.60:42911 requested a full tablet report, sending...
I20260620 22:59:36.932536 10114 heartbeater.cc:511] Master 127.2.237.62:36895 requested a full tablet report, sending...
I20260620 22:59:36.932775 10116 heartbeater.cc:347] Connected to a master server at 127.2.237.61:38995
I20260620 22:59:36.932891 10116 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:36.933938 10116 heartbeater.cc:511] Master 127.2.237.61:38995 requested a full tablet report, sending...
I20260620 22:59:36.934528 9467 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:36.934561 9149 ts_manager.cc:194] Re-registered known tserver with Master: 8154a02194f741cbb6596035cec6fcf0 (127.2.237.1:41217)
I20260620 22:59:36.935683 9149 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.237.1@KRBTEST.COM'} at 127.2.237.1:45697
I20260620 22:59:36.935917 9935 ts_manager.cc:194] Registered new tserver with Master: 8154a02194f741cbb6596035cec6fcf0 (127.2.237.1:41217)
I20260620 22:59:36.937395 9747 ts_manager.cc:194] Registered new tserver with Master: 8154a02194f741cbb6596035cec6fcf0 (127.2.237.1:41217)
I20260620 22:59:37.020398 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 9344
I20260620 22:59:37.036578 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:33799
--local_ip_for_outbound_sockets=127.2.237.2
--tserver_master_addrs=127.2.237.62:36895
--webserver_port=38341
--webserver_interface=127.2.237.2
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.2.keytab
--principal=oryx/127.2.237.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:39019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.2.237.62:36895,127.2.237.61:38995
--tserver_master_addrs=127.2.237.62:36895,127.2.237.61:38995,127.2.237.60:42911 with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 22:59:37.156105 9894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "988aa53a4d75433ba11c4b68df409c91" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: NON_VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: true } } }
I20260620 22:59:37.156512 9894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:37.158444 10128 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91: attempting to promote NON_VOTER c582ab7db2fb468aaa0862be09f6e534 to VOTER
I20260620 22:59:37.159195 10128 consensus_queue.cc:237] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: false } }
I20260620 22:59:37.161155 9945 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [term 1 LEARNER]: Refusing update from remote peer 988aa53a4d75433ba11c4b68df409c91: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260620 22:59:37.161243 9756 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 1 FOLLOWER]: Refusing update from remote peer 988aa53a4d75433ba11c4b68df409c91: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260620 22:59:37.161784 9894 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260620 22:59:37.162168 10127 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260620 22:59:37.164475 9894 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, c582ab7db2fb468aaa0862be09f6e534 (127.2.237.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: false } } }
I20260620 22:59:37.165025 9756 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, c582ab7db2fb468aaa0862be09f6e534 (127.2.237.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: false } } }
I20260620 22:59:37.168754 9894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "988aa53a4d75433ba11c4b68df409c91" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: false } } }
I20260620 22:59:37.169073 9894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:37.169407 10128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "988aa53a4d75433ba11c4b68df409c91" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: false } } }
I20260620 22:59:37.169658 10128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:37.171167 9894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "988aa53a4d75433ba11c4b68df409c91" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: false } } }
I20260620 22:59:37.171480 9894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:37.171880 9945 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, c582ab7db2fb468aaa0862be09f6e534 (127.2.237.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: false } } }
I20260620 22:59:37.178504 10131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "988aa53a4d75433ba11c4b68df409c91" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: false } } }
I20260620 22:59:37.178988 10131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [sys.catalog]: This master's current role is: FOLLOWER
I20260620 22:59:37.179411 10132 sys_catalog.cc:455] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "988aa53a4d75433ba11c4b68df409c91" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: false } } }
I20260620 22:59:37.179669 10132 sys_catalog.cc:458] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [sys.catalog]: This master's current role is: FOLLOWER
W20260620 22:59:37.373917 10126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:37.374208 10126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:37.374363 10126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:37.382012 10126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:37.382158 10126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:37.382329 10126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 22:59:37.392936 10126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:39019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.2.keytab
--principal=oryx/127.2.237.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:33799
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=38341
--tserver_master_addrs=127.2.237.62:36895,127.2.237.61:38995,127.2.237.60:42911
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:37.394268 10126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:37.395656 10126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:37.409287 10138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:37.409483 10140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:37.409843 10137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:37.412038 10126 server_base.cc:1034] running on GCE node
I20260620 22:59:37.412647 10126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:37.413615 10126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:37.414837 10126 hybrid_clock.cc:648] HybridClock initialized: now 1781996377414778 us; error 51 us; skew 500 ppm
I20260620 22:59:37.415148 10126 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:37 dist-test-slave-f63b krb5kdc[9104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996377, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:37.418675 10126 init.cc:377] Logged in from keytab as oryx/127.2.237.2@KRBTEST.COM (short username oryx)
I20260620 22:59:37.421725 10126 webserver.cc:460] Webserver started at http://127.2.237.2:38341/ using document root <none> and password file <none>
I20260620 22:59:37.422441 10126 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:37.422613 10126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:37.428002 10126 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260620 22:59:37.430919 10147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:37.432402 10126 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260620 22:59:37.432555 10126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "d3ffb93de8d841159da80de89f7171e4"
format_stamp: "Formatted at 2026-06-20 22:59:34 on dist-test-slave-f63b"
server_key: "64a2476ffc991825fa208bfd04090737"
server_key_iv: "170d7e6d4b98a2e5989ee0a198d97b8d"
server_key_version: "encryptionkey@0"
I20260620 22:59:37.433483 10126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:37.450162 10126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:37.454459 10126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:37.454772 10126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:37.455744 10126 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:59:37.457157 10126 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:37.457260 10126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:37.457386 10126 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:37.457479 10126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:37.489136 10126 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:33799
I20260620 22:59:37.489185 10259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:33799 every 8 connection(s)
I20260620 22:59:37.490717 10126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 22:59:37.497973 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 10126
I20260620 22:59:37.498327 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 9477
Jun 20 22:59:37 dist-test-slave-f63b krb5kdc[9104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996377, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM
Jun 20 22:59:37 dist-test-slave-f63b krb5kdc[9104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996377, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.60@KRBTEST.COM
Jun 20 22:59:37 dist-test-slave-f63b krb5kdc[9104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996377, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.61@KRBTEST.COM
I20260620 22:59:37.520380 10264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:37.493757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:34173 (local address 127.2.237.62:36895)
0620 22:59:37.494234 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:37.494240 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:37.495401 (+ 1161us) server_negotiation.cc:407] Connection header received
0620 22:59:37.500439 (+ 5038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:37.500449 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:37.500558 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:37.500692 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:37.503042 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:37.504519 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:37.506857 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:37.507364 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:37.507507 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:37.512705 (+ 5198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:37.512712 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:37.512717 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:37.512764 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:37.516840 (+ 4076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:37.517665 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:37.517670 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:37.517674 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:37.517805 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:37.518326 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:37.518331 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:37.518335 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:37.518736 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:37.518903 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:37.519826 (+ 923us) server_negotiation.cc:299] Negotiation successful
0620 22:59:37.520027 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":208,"threads_started":1}
Jun 20 22:59:37 dist-test-slave-f63b krb5kdc[9104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.237.3@KRBTEST.COM
I20260620 22:59:37.521720 10266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:37.494366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.2.237.2:34173)
0620 22:59:37.495241 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:37.495294 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:37.500293 (+ 4999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:37.501010 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:37.501024 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:37.501890 (+ 866us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:37.502867 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:37.502893 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:37.504702 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:37.504710 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:37.506718 (+ 2008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:37.506732 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:37.507597 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:37.507606 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:37.508057 (+ 451us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:37.508898 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:37.508939 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:37.512495 (+ 3556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:37.517007 (+ 4512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:37.517016 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:37.517036 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:37.517546 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:37.517911 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:37.517917 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:37.517920 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:37.518223 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:37.518872 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:37.518880 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:37.519304 (+ 424us) client_negotiation.cc:769] Sending connection context
0620 22:59:37.519738 (+ 434us) client_negotiation.cc:240] Negotiation successful
0620 22:59:37.520253 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"mutex_wait_us":1,"thread_start_us":2796,"threads_started":1}
I20260620 22:59:37.523260 10260 heartbeater.cc:347] Connected to a master server at 127.2.237.62:36895
I20260620 22:59:37.523604 10260 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:37.524538 10260 heartbeater.cc:511] Master 127.2.237.62:36895 requested a full tablet report, sending...
I20260620 22:59:37.526140 9149 ts_manager.cc:194] Re-registered known tserver with Master: d3ffb93de8d841159da80de89f7171e4 (127.2.237.2:33799)
I20260620 22:59:37.525858 10267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:37.495092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:42911 (local address 127.2.237.2:56293)
0620 22:59:37.495861 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:37.495887 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:37.498471 (+ 2584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:37.505496 (+ 7025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:37.505501 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:37.506128 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:37.506418 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:37.506429 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:37.507913 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:37.507918 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:37.509591 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:37.509609 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:37.510627 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:37.510634 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:37.510811 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:37.511431 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:37.511453 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:37.514850 (+ 3397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:37.521549 (+ 6699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:37.521554 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:37.521558 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:37.522546 (+ 988us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:37.523502 (+ 956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:37.523509 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:37.523518 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:37.523674 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:37.524439 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:37.524443 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:37.524583 (+ 140us) client_negotiation.cc:769] Sending connection context
0620 22:59:37.524783 (+ 200us) client_negotiation.cc:240] Negotiation successful
0620 22:59:37.525073 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":1124,"threads_started":1}
I20260620 22:59:37.527374 9149 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.237.2@KRBTEST.COM'} at 127.2.237.2:34173
I20260620 22:59:37.527470 10268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:37.495539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:56293 (local address 127.2.237.60:42911)
0620 22:59:37.505099 (+ 9560us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:37.505108 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:37.505130 (+ 22us) server_negotiation.cc:407] Connection header received
0620 22:59:37.505216 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:37.505224 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:37.505322 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:37.505455 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:37.506624 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:37.507754 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:37.509784 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:37.510418 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:37.510551 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:37.515063 (+ 4512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:37.515071 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:37.515078 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:37.515131 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:37.521396 (+ 6265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:37.522712 (+ 1316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:37.522719 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:37.522722 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:37.523381 (+ 659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:37.523808 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:37.523813 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:37.523817 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:37.524234 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:37.524364 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:37.525518 (+ 1154us) server_negotiation.cc:299] Negotiation successful
0620 22:59:37.527080 (+ 1562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9400,"thread_start_us":9246,"threads_started":1}
I20260620 22:59:37.528335 10269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:37.495976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:38995 (local address 127.2.237.2:39749)
0620 22:59:37.496754 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:37.496775 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:37.500198 (+ 3423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:37.507616 (+ 7418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:37.507621 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:37.508295 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:37.508965 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:37.508978 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:37.511625 (+ 2647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:37.511630 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:37.513617 (+ 1987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:37.513628 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:37.515174 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:37.515181 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:37.515367 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:37.516063 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:37.516084 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:37.518888 (+ 2804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:37.525624 (+ 6736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:37.525629 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:37.525632 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:37.526150 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:37.526542 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:37.526546 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:37.526550 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:37.526736 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:37.527349 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:37.527355 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:37.527488 (+ 133us) client_negotiation.cc:769] Sending connection context
0620 22:59:37.527623 (+ 135us) client_negotiation.cc:240] Negotiation successful
0620 22:59:37.527909 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":1104,"threads_started":1}
I20260620 22:59:37.528999 10261 heartbeater.cc:347] Connected to a master server at 127.2.237.60:42911
I20260620 22:59:37.529136 10261 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:37.529387 10271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:37.499443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53108 (local address 127.2.237.3:34669)
0620 22:59:37.505900 (+ 6457us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:37.505911 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:37.505944 (+ 33us) server_negotiation.cc:407] Connection header received
0620 22:59:37.506032 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:37.506043 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:37.506347 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:37.509559 (+ 3212us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:37.509799 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:37.511368 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:37.513998 (+ 2630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:37.514931 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:37.515629 (+ 698us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:37.522299 (+ 6670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:37.522312 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:37.522328 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:37.522365 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:37.526303 (+ 3938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:37.527220 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:37.527228 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:37.527232 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:37.527383 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:37.527812 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:37.527818 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:37.527821 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:37.528129 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:37.528430 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:37.528749 (+ 319us) server_negotiation.cc:299] Negotiation successful
0620 22:59:37.528993 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6251,"thread_start_us":3326,"threads_started":1}
I20260620 22:59:37.529588 10261 heartbeater.cc:511] Master 127.2.237.60:42911 requested a full tablet report, sending...
I20260620 22:59:37.530534 9600 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:37.530670 9935 ts_manager.cc:194] Registered new tserver with Master: d3ffb93de8d841159da80de89f7171e4 (127.2.237.2:33799)
I20260620 22:59:37.532209 10265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:37.493850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:39749 (local address 127.2.237.61:38995)
0620 22:59:37.507253 (+ 13403us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:37.507259 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:37.507278 (+ 19us) server_negotiation.cc:407] Connection header received
0620 22:59:37.507332 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:37.507337 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:37.507436 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:37.509123 (+ 1687us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:37.509309 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:37.511438 (+ 2129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:37.514297 (+ 2859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:37.514988 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:37.515696 (+ 708us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:37.519051 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:37.519057 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:37.519062 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:37.519166 (+ 104us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:37.525419 (+ 6253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:37.526279 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:37.526284 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:37.526287 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:37.526441 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:37.526870 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:37.526877 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:37.526881 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:37.527212 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:37.531494 (+ 4282us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:37.531629 (+ 135us) server_negotiation.cc:299] Negotiation successful
0620 22:59:37.531859 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13290,"thread_start_us":13111,"threads_started":1}
I20260620 22:59:37.533712 10262 heartbeater.cc:347] Connected to a master server at 127.2.237.61:38995
I20260620 22:59:37.533844 10262 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:37.534273 10262 heartbeater.cc:511] Master 127.2.237.61:38995 requested a full tablet report, sending...
I20260620 22:59:37.535315 9747 ts_manager.cc:194] Registered new tserver with Master: d3ffb93de8d841159da80de89f7171e4 (127.2.237.2:33799)
I20260620 22:59:37.614455 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 9477
I20260620 22:59:37.631016 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:34669
--local_ip_for_outbound_sockets=127.2.237.3
--tserver_master_addrs=127.2.237.62:36895
--webserver_port=43921
--webserver_interface=127.2.237.3
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.3.keytab
--principal=oryx/127.2.237.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:39019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.2.237.62:36895,127.2.237.61:38995
--tserver_master_addrs=127.2.237.62:36895,127.2.237.61:38995,127.2.237.60:42911 with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 22:59:37.940304 10114 heartbeater.cc:503] Master 127.2.237.62:36895 was elected leader, sending a full tablet report...
W20260620 22:59:37.958214 10273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:37.958523 10273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:37.958707 10273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:37.966180 10273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:37.966338 10273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:37.966512 10273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 22:59:37.976500 10273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:39019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.3.keytab
--principal=oryx/127.2.237.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:34669
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=43921
--tserver_master_addrs=127.2.237.62:36895,127.2.237.61:38995,127.2.237.60:42911
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:37.977804 10273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:37.979118 10273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:37.992933 10281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:37.992960 10279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:37.993027 10278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:37.993636 10273 server_base.cc:1034] running on GCE node
I20260620 22:59:37.994283 10273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:37.995226 10273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:37.996456 10273 hybrid_clock.cc:648] HybridClock initialized: now 1781996377996403 us; error 46 us; skew 500 ppm
I20260620 22:59:37.996809 10273 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:37 dist-test-slave-f63b krb5kdc[9104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996377, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:37.999892 10273 init.cc:377] Logged in from keytab as oryx/127.2.237.3@KRBTEST.COM (short username oryx)
I20260620 22:59:38.001379 10273 webserver.cc:460] Webserver started at http://127.2.237.3:43921/ using document root <none> and password file <none>
I20260620 22:59:38.002094 10273 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:38.002251 10273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:38.007601 10273 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260620 22:59:38.010174 10288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:38.011570 10273 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260620 22:59:38.011734 10273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "3846f009be844b4ab3849d5ad442e6cb"
format_stamp: "Formatted at 2026-06-20 22:59:34 on dist-test-slave-f63b"
server_key: "e9c50b728a8bca74a40f96f9bcdc647f"
server_key_iv: "8a4f5a2364110dc0e423f95efeb403be"
server_key_version: "encryptionkey@0"
I20260620 22:59:38.012683 10273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:38.037951 10273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:38.042459 10273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:38.042776 10273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:38.043882 10273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:59:38.045228 10273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:38.045343 10273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:38.045497 10273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:38.045598 10273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:38.079356 10273 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:34669
I20260620 22:59:38.079406 10400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:34669 every 8 connection(s)
I20260620 22:59:38.080971 10273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 22:59:38.082093 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 10273
Jun 20 22:59:38 dist-test-slave-f63b krb5kdc[9104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996377, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.60@KRBTEST.COM
Jun 20 22:59:38 dist-test-slave-f63b krb5kdc[9104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996377, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.61@KRBTEST.COM
Jun 20 22:59:38 dist-test-slave-f63b krb5kdc[9104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996377, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM
I20260620 22:59:38.138737 10409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:38.085268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:46085 (local address 127.2.237.60:42911)
0620 22:59:38.107707 (+ 22439us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:38.107716 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:38.112675 (+ 4959us) server_negotiation.cc:407] Connection header received
0620 22:59:38.112760 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:38.112769 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:38.112896 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:38.113009 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:38.117651 (+ 4642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.119240 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.122751 (+ 3511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.123402 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.123501 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:38.128252 (+ 4751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:38.128260 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:38.128266 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:38.128308 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:38.132968 (+ 4660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:38.135665 (+ 2697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:38.135672 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:38.135681 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:38.135858 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:38.136396 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:38.136404 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:38.136410 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:38.136772 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:38.136955 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:38.138072 (+ 1117us) server_negotiation.cc:299] Negotiation successful
0620 22:59:38.138286 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22294,"thread_start_us":22754,"threads_started":1}
I20260620 22:59:38.141809 10407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:38.084694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:42911 (local address 127.2.237.3:46085)
0620 22:59:38.108836 (+ 24142us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:38.108900 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:38.112117 (+ 3217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:38.114728 (+ 2611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:38.114746 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:38.115755 (+ 1009us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:38.117403 (+ 1648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:38.117433 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.119478 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.119486 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:38.122555 (+ 3069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:38.122590 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.123841 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.123848 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:38.124108 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:38.124917 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:38.124952 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:38.127998 (+ 3046us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:38.133517 (+ 5519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:38.133527 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:38.133547 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:38.135520 (+ 1973us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:38.136001 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:38.136006 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:38.136011 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:38.136266 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:38.137053 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:38.137062 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:38.137553 (+ 491us) client_negotiation.cc:769] Sending connection context
0620 22:59:38.138949 (+ 1396us) client_negotiation.cc:240] Negotiation successful
0620 22:59:38.139548 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23710,"thread_start_us":29342,"threads_started":1}
I20260620 22:59:38.142776 10408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:38.085151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:38995 (local address 127.2.237.3:47959)
0620 22:59:38.113592 (+ 28441us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:38.113612 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:38.113714 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:38.121213 (+ 7499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:38.121218 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:38.121984 (+ 766us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:38.122297 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:38.122308 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.124383 (+ 2075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.124390 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:38.125984 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:38.125999 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.129411 (+ 3412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.129421 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:38.129625 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:38.130150 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:38.130603 (+ 453us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:38.134440 (+ 3837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:38.139224 (+ 4784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:38.139229 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:38.139233 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:38.139802 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:38.140292 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:38.140297 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:38.140300 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:38.140426 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:38.141048 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:38.141054 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:38.141320 (+ 266us) client_negotiation.cc:769] Sending connection context
0620 22:59:38.141476 (+ 156us) client_negotiation.cc:240] Negotiation successful
0620 22:59:38.141786 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28323,"spinlock_wait_cycles":34048,"thread_start_us":28187,"threads_started":1}
I20260620 22:59:38.143836 10410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:38.085557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:47959 (local address 127.2.237.61:38995)
0620 22:59:38.120903 (+ 35346us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:38.120911 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:38.120929 (+ 18us) server_negotiation.cc:407] Connection header received
0620 22:59:38.120995 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:38.121000 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:38.121073 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:38.121188 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:38.123130 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.124210 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.127963 (+ 3753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.128646 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.129927 (+ 1281us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:38.134688 (+ 4761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:38.134694 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:38.134704 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:38.134764 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:38.139076 (+ 4312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:38.139980 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:38.139986 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:38.139990 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:38.140158 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:38.140559 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:38.140564 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:38.140566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:38.140884 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:38.141056 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:38.143192 (+ 2136us) server_negotiation.cc:299] Negotiation successful
0620 22:59:38.143429 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35230,"thread_start_us":36237,"threads_started":1}
I20260620 22:59:38.144058 10402 heartbeater.cc:347] Connected to a master server at 127.2.237.60:42911
I20260620 22:59:38.144399 10402 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:38.144651 10403 heartbeater.cc:347] Connected to a master server at 127.2.237.61:38995
I20260620 22:59:38.144785 10403 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:38.145483 10402 heartbeater.cc:511] Master 127.2.237.60:42911 requested a full tablet report, sending...
I20260620 22:59:38.145515 10403 heartbeater.cc:511] Master 127.2.237.61:38995 requested a full tablet report, sending...
I20260620 22:59:38.147313 9747 ts_manager.cc:194] Registered new tserver with Master: 3846f009be844b4ab3849d5ad442e6cb (127.2.237.3:34669)
I20260620 22:59:38.147702 10411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:38.086615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.2.237.3:43649)
0620 22:59:38.120197 (+ 33582us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:38.120221 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:38.120347 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:38.121379 (+ 1032us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:38.121386 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:38.122691 (+ 1305us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:38.122947 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:38.122957 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.126146 (+ 3189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.126150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:38.127732 (+ 1582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:38.127745 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.128842 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.128848 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:38.129051 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:38.132162 (+ 3111us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:38.132186 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:38.135024 (+ 2838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:38.139683 (+ 4659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:38.139697 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:38.139701 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:38.140164 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:38.141128 (+ 964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:38.141133 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:38.141137 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:38.141319 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:38.144375 (+ 3056us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:38.144381 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:38.144552 (+ 171us) client_negotiation.cc:769] Sending connection context
0620 22:59:38.145185 (+ 633us) client_negotiation.cc:240] Negotiation successful
0620 22:59:38.145488 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33467,"thread_start_us":33802,"threads_started":1}
I20260620 22:59:38.148129 9935 ts_manager.cc:194] Registered new tserver with Master: 3846f009be844b4ab3849d5ad442e6cb (127.2.237.3:34669)
I20260620 22:59:38.150175 10406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:38.084027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:43649 (local address 127.2.237.62:36895)
0620 22:59:38.100302 (+ 16275us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:38.100310 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:38.120308 (+ 19998us) server_negotiation.cc:407] Connection header received
0620 22:59:38.120470 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:38.120480 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:38.120603 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:38.120743 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:38.123093 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.124625 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.127890 (+ 3265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.128673 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.128795 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:38.135225 (+ 6430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:38.135234 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:38.135242 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:38.135292 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:38.139571 (+ 4279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:38.140306 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:38.140312 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:38.140316 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:38.140472 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:38.141464 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:38.141469 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:38.141472 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:38.141815 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:38.141990 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:38.149515 (+ 7525us) server_negotiation.cc:299] Negotiation successful
0620 22:59:38.149785 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16111,"thread_start_us":16269,"threads_started":1}
I20260620 22:59:38.151023 10401 heartbeater.cc:347] Connected to a master server at 127.2.237.62:36895
I20260620 22:59:38.151147 10401 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:38.151527 10401 heartbeater.cc:511] Master 127.2.237.62:36895 requested a full tablet report, sending...
I20260620 22:59:38.152467 9149 ts_manager.cc:194] Re-registered known tserver with Master: 3846f009be844b4ab3849d5ad442e6cb (127.2.237.3:34669)
I20260620 22:59:38.153764 9149 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.237.3@KRBTEST.COM'} at 127.2.237.3:43649
I20260620 22:59:38.160030 10415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:38.102177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56904 (local address 127.2.237.62:36895)
0620 22:59:38.114138 (+ 11961us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:38.114144 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:38.135786 (+ 21642us) server_negotiation.cc:407] Connection header received
0620 22:59:38.135954 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:38.135959 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:38.136038 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:38.136139 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:38.143114 (+ 6975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.144180 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.148033 (+ 3853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.148748 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.148847 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:38.153095 (+ 4248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:38.153101 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:38.153108 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:38.153138 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:38.156850 (+ 3712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:38.157585 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:38.157591 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:38.157595 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:38.157791 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:38.158256 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:38.158262 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:38.158265 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:38.158683 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:38.158955 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:38.159338 (+ 383us) server_negotiation.cc:299] Negotiation successful
0620 22:59:38.159605 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11839,"thread_start_us":11725,"threads_started":1}
I20260620 22:59:38.163072 10416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:38.102312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42752 (local address 127.2.237.60:42911)
0620 22:59:38.108025 (+ 5713us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:38.108033 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:38.128627 (+ 20594us) server_negotiation.cc:407] Connection header received
0620 22:59:38.128931 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:38.128938 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:38.129096 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:38.129204 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:38.139763 (+ 10559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.140893 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.148178 (+ 7285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.148925 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.149022 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:38.155444 (+ 6422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:38.155451 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:38.155456 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:38.155498 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:38.159781 (+ 4283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:38.160534 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:38.160541 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:38.160545 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:38.160709 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:38.161210 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:38.161216 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:38.161219 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:38.161653 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:38.161901 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:38.162507 (+ 606us) server_negotiation.cc:299] Negotiation successful
0620 22:59:38.162774 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5595,"thread_start_us":5521,"threads_started":1}
I20260620 22:59:38.164803 10418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:38.108361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43226 (local address 127.2.237.61:38995)
0620 22:59:38.135102 (+ 26741us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:38.135108 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:38.142782 (+ 7674us) server_negotiation.cc:407] Connection header received
0620 22:59:38.144246 (+ 1464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:38.144251 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:38.144325 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:38.144405 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:38.147199 (+ 2794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.149371 (+ 2172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.151991 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.152593 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.152683 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:38.156566 (+ 3883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:38.156573 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:38.156577 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:38.156616 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:38.160265 (+ 3649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:38.161066 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:38.161071 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:38.161075 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:38.161255 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:38.161767 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:38.161772 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:38.161775 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:38.162065 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:38.162339 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:38.164145 (+ 1806us) server_negotiation.cc:299] Negotiation successful
0620 22:59:38.164370 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26540,"thread_start_us":26752,"threads_started":1}
I20260620 22:59:38.164484 9149 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56904:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 22:59:38.172001 9149 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 22:59:38.213311 10434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:38.195880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47086 (local address 127.2.237.1:41217)
0620 22:59:38.205577 (+ 9697us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:38.205584 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:38.205610 (+ 26us) server_negotiation.cc:407] Connection header received
0620 22:59:38.205694 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:38.205700 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:38.205913 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:38.206140 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:59:38.206826 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.208491 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.211270 (+ 2779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.212180 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.212344 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:38.212584 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:38.212823 (+ 239us) server_negotiation.cc:299] Negotiation successful
0620 22:59:38.212999 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9526,"thread_start_us":9405,"threads_started":1}
I20260620 22:59:38.213390 10433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:38.195603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:41217 (local address 127.0.0.1:47086)
0620 22:59:38.197406 (+ 1803us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:38.197431 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:38.197578 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:38.206115 (+ 8537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:38.206120 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:38.206147 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:59:38.206602 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:38.206620 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.208663 (+ 2043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.208667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:38.211125 (+ 2458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:38.211136 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.212415 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.212423 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:38.212606 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:38.212640 (+ 34us) client_negotiation.cc:769] Sending connection context
0620 22:59:38.212776 (+ 136us) client_negotiation.cc:240] Negotiation successful
0620 22:59:38.212957 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1633,"mutex_wait_us":25,"thread_start_us":1462,"threads_started":1}
I20260620 22:59:38.215785 10435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:38.196653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:33799 (local address 127.0.0.1:33920)
0620 22:59:38.207336 (+ 10683us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:38.207355 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:38.207513 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:38.208148 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:38.208153 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:38.208177 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:59:38.208469 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:38.208477 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.210757 (+ 2280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.210765 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:38.212951 (+ 2186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:38.212969 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.214998 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.215004 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:38.215167 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:38.215200 (+ 33us) client_negotiation.cc:769] Sending connection context
0620 22:59:38.215280 (+ 80us) client_negotiation.cc:240] Negotiation successful
0620 22:59:38.215411 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10556,"thread_start_us":10415,"threads_started":1}
I20260620 22:59:38.217025 10436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:38.197253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33920 (local address 127.2.237.2:33799)
0620 22:59:38.206875 (+ 9622us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:38.206886 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:38.207516 (+ 630us) server_negotiation.cc:407] Connection header received
0620 22:59:38.207665 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:38.207678 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:38.207951 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:38.208159 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:59:38.208625 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.210559 (+ 1934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.213133 (+ 2574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.214814 (+ 1681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.214953 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:38.216441 (+ 1488us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:38.216512 (+ 71us) server_negotiation.cc:299] Negotiation successful
0620 22:59:38.216691 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9471,"thread_start_us":9391,"threads_started":1}
I20260620 22:59:38.217240 10438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:38.197841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53120 (local address 127.2.237.3:34669)
0620 22:59:38.198651 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:38.198664 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:38.208117 (+ 9453us) server_negotiation.cc:407] Connection header received
0620 22:59:38.208392 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:38.208400 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:38.208669 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:38.208806 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:59:38.209831 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.212058 (+ 2227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.214528 (+ 2470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.216108 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.216247 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:38.216438 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:38.216721 (+ 283us) server_negotiation.cc:299] Negotiation successful
0620 22:59:38.216878 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":336,"threads_started":1}
I20260620 22:59:38.217250 10437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:38.197377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:34669 (local address 127.0.0.1:53120)
0620 22:59:38.207995 (+ 10618us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:38.208012 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:38.208224 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:38.209102 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:38.209106 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:38.209131 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:59:38.209616 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:38.209626 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.212230 (+ 2604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.212235 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:38.214365 (+ 2130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:38.214379 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.216367 (+ 1988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.216374 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:38.216551 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:38.216580 (+ 29us) client_negotiation.cc:769] Sending connection context
0620 22:59:38.216663 (+ 83us) client_negotiation.cc:240] Negotiation successful
0620 22:59:38.216772 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10489,"mutex_wait_us":3,"thread_start_us":10366,"threads_started":1}
I20260620 22:59:38.217569 10049 tablet_service.cc:1467] Processing CreateTablet for tablet 37a9608a3fcd4226a9082d7fbf81a99c (DEFAULT_TABLE table=test-table [id=5a7b09a07f014288b2959f2d598e3b0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:59:38.219151 10049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37a9608a3fcd4226a9082d7fbf81a99c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:38.220870 10195 tablet_service.cc:1467] Processing CreateTablet for tablet 37a9608a3fcd4226a9082d7fbf81a99c (DEFAULT_TABLE table=test-table [id=5a7b09a07f014288b2959f2d598e3b0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:59:38.222167 10195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37a9608a3fcd4226a9082d7fbf81a99c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:38.221937 10336 tablet_service.cc:1467] Processing CreateTablet for tablet 37a9608a3fcd4226a9082d7fbf81a99c (DEFAULT_TABLE table=test-table [id=5a7b09a07f014288b2959f2d598e3b0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:59:38.223191 10336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37a9608a3fcd4226a9082d7fbf81a99c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:38.230813 10439 tablet_bootstrap.cc:492] T 37a9608a3fcd4226a9082d7fbf81a99c P 8154a02194f741cbb6596035cec6fcf0: Bootstrap starting.
I20260620 22:59:38.234994 10439 tablet_bootstrap.cc:654] T 37a9608a3fcd4226a9082d7fbf81a99c P 8154a02194f741cbb6596035cec6fcf0: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:38.235971 10439 log.cc:824] T 37a9608a3fcd4226a9082d7fbf81a99c P 8154a02194f741cbb6596035cec6fcf0: Log is configured to *not* fsync() on all Append() calls
I20260620 22:59:38.238032 10442 tablet_bootstrap.cc:492] T 37a9608a3fcd4226a9082d7fbf81a99c P 3846f009be844b4ab3849d5ad442e6cb: Bootstrap starting.
I20260620 22:59:38.239256 10441 tablet_bootstrap.cc:492] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4: Bootstrap starting.
I20260620 22:59:38.239276 10439 tablet_bootstrap.cc:492] T 37a9608a3fcd4226a9082d7fbf81a99c P 8154a02194f741cbb6596035cec6fcf0: No bootstrap required, opened a new log
I20260620 22:59:38.239980 10439 ts_tablet_manager.cc:1397] T 37a9608a3fcd4226a9082d7fbf81a99c P 8154a02194f741cbb6596035cec6fcf0: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260620 22:59:38.241943 10441 tablet_bootstrap.cc:654] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:38.241979 10442 tablet_bootstrap.cc:654] T 37a9608a3fcd4226a9082d7fbf81a99c P 3846f009be844b4ab3849d5ad442e6cb: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:38.243196 10441 log.cc:824] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4: Log is configured to *not* fsync() on all Append() calls
I20260620 22:59:38.243268 10442 log.cc:824] T 37a9608a3fcd4226a9082d7fbf81a99c P 3846f009be844b4ab3849d5ad442e6cb: Log is configured to *not* fsync() on all Append() calls
I20260620 22:59:38.245571 10441 tablet_bootstrap.cc:492] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4: No bootstrap required, opened a new log
I20260620 22:59:38.245812 10441 ts_tablet_manager.cc:1397] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260620 22:59:38.250160 10442 tablet_bootstrap.cc:492] T 37a9608a3fcd4226a9082d7fbf81a99c P 3846f009be844b4ab3849d5ad442e6cb: No bootstrap required, opened a new log
I20260620 22:59:38.250412 10442 ts_tablet_manager.cc:1397] T 37a9608a3fcd4226a9082d7fbf81a99c P 3846f009be844b4ab3849d5ad442e6cb: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.003s
I20260620 22:59:38.252205 10439 raft_consensus.cc:348] T 37a9608a3fcd4226a9082d7fbf81a99c P 8154a02194f741cbb6596035cec6fcf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8154a02194f741cbb6596035cec6fcf0" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41217 } } peers { permanent_uuid: "3846f009be844b4ab3849d5ad442e6cb" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34669 } } peers { permanent_uuid: "d3ffb93de8d841159da80de89f7171e4" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33799 } }
I20260620 22:59:38.252627 10439 raft_consensus.cc:374] T 37a9608a3fcd4226a9082d7fbf81a99c P 8154a02194f741cbb6596035cec6fcf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:38.252724 10439 raft_consensus.cc:729] T 37a9608a3fcd4226a9082d7fbf81a99c P 8154a02194f741cbb6596035cec6fcf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8154a02194f741cbb6596035cec6fcf0, State: Initialized, Role: FOLLOWER
I20260620 22:59:38.253453 10439 consensus_queue.cc:260] T 37a9608a3fcd4226a9082d7fbf81a99c P 8154a02194f741cbb6596035cec6fcf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8154a02194f741cbb6596035cec6fcf0" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41217 } } peers { permanent_uuid: "3846f009be844b4ab3849d5ad442e6cb" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34669 } } peers { permanent_uuid: "d3ffb93de8d841159da80de89f7171e4" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33799 } }
I20260620 22:59:38.253944 10441 raft_consensus.cc:348] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8154a02194f741cbb6596035cec6fcf0" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41217 } } peers { permanent_uuid: "3846f009be844b4ab3849d5ad442e6cb" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34669 } } peers { permanent_uuid: "d3ffb93de8d841159da80de89f7171e4" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33799 } }
I20260620 22:59:38.254328 10441 raft_consensus.cc:374] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:38.254426 10441 raft_consensus.cc:729] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3ffb93de8d841159da80de89f7171e4, State: Initialized, Role: FOLLOWER
I20260620 22:59:38.255133 10441 consensus_queue.cc:260] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8154a02194f741cbb6596035cec6fcf0" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41217 } } peers { permanent_uuid: "3846f009be844b4ab3849d5ad442e6cb" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34669 } } peers { permanent_uuid: "d3ffb93de8d841159da80de89f7171e4" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33799 } }
I20260620 22:59:38.261277 10439 ts_tablet_manager.cc:1428] T 37a9608a3fcd4226a9082d7fbf81a99c P 8154a02194f741cbb6596035cec6fcf0: Time spent starting tablet: real 0.021s user 0.013s sys 0.001s
I20260620 22:59:38.264875 10260 heartbeater.cc:503] Master 127.2.237.62:36895 was elected leader, sending a full tablet report...
I20260620 22:59:38.265612 10441 ts_tablet_manager.cc:1428] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4: Time spent starting tablet: real 0.020s user 0.012s sys 0.008s
I20260620 22:59:38.268582 10442 raft_consensus.cc:348] T 37a9608a3fcd4226a9082d7fbf81a99c P 3846f009be844b4ab3849d5ad442e6cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8154a02194f741cbb6596035cec6fcf0" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41217 } } peers { permanent_uuid: "3846f009be844b4ab3849d5ad442e6cb" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34669 } } peers { permanent_uuid: "d3ffb93de8d841159da80de89f7171e4" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33799 } }
I20260620 22:59:38.268978 10442 raft_consensus.cc:374] T 37a9608a3fcd4226a9082d7fbf81a99c P 3846f009be844b4ab3849d5ad442e6cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:38.269071 10442 raft_consensus.cc:729] T 37a9608a3fcd4226a9082d7fbf81a99c P 3846f009be844b4ab3849d5ad442e6cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3846f009be844b4ab3849d5ad442e6cb, State: Initialized, Role: FOLLOWER
I20260620 22:59:38.269681 10442 consensus_queue.cc:260] T 37a9608a3fcd4226a9082d7fbf81a99c P 3846f009be844b4ab3849d5ad442e6cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8154a02194f741cbb6596035cec6fcf0" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41217 } } peers { permanent_uuid: "3846f009be844b4ab3849d5ad442e6cb" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34669 } } peers { permanent_uuid: "d3ffb93de8d841159da80de89f7171e4" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33799 } }
I20260620 22:59:38.271235 10401 heartbeater.cc:503] Master 127.2.237.62:36895 was elected leader, sending a full tablet report...
I20260620 22:59:38.271265 10442 ts_tablet_manager.cc:1428] T 37a9608a3fcd4226a9082d7fbf81a99c P 3846f009be844b4ab3849d5ad442e6cb: Time spent starting tablet: real 0.021s user 0.014s sys 0.001s
I20260620 22:59:38.322376 10446 raft_consensus.cc:482] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 22:59:38.322670 10446 raft_consensus.cc:504] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8154a02194f741cbb6596035cec6fcf0" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41217 } } peers { permanent_uuid: "3846f009be844b4ab3849d5ad442e6cb" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34669 } } peers { permanent_uuid: "d3ffb93de8d841159da80de89f7171e4" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33799 } }
I20260620 22:59:38.324293 10446 leader_election.cc:288] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8154a02194f741cbb6596035cec6fcf0 (127.2.237.1:41217), 3846f009be844b4ab3849d5ad442e6cb (127.2.237.3:34669)
I20260620 22:59:38.331892 10448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:38.324561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:41217 (local address 127.2.237.2:58657)
0620 22:59:38.325197 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:38.325217 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:38.325352 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:38.325760 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:38.325765 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:38.325791 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:59:38.326122 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:38.326131 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.327405 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.327409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:38.329463 (+ 2054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:38.329478 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.330924 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.330928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:38.331047 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:38.331071 (+ 24us) client_negotiation.cc:769] Sending connection context
0620 22:59:38.331146 (+ 75us) client_negotiation.cc:240] Negotiation successful
0620 22:59:38.331231 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":320,"threads_started":1}
I20260620 22:59:38.332223 10449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:38.325298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:34669 (local address 127.2.237.2:39655)
0620 22:59:38.325944 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:38.325958 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:38.326103 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:38.326413 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:38.326417 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:38.326437 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:59:38.326817 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:38.326829 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.327952 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.327958 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:38.329856 (+ 1898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:38.329866 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.331433 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.331440 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:38.331605 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:38.331637 (+ 32us) client_negotiation.cc:769] Sending connection context
0620 22:59:38.331715 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 22:59:38.331794 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":386,"threads_started":1}
I20260620 22:59:38.332240 10438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:38.325454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:39655 (local address 127.2.237.3:34669)
0620 22:59:38.325638 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:38.325646 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:38.326046 (+ 400us) server_negotiation.cc:407] Connection header received
0620 22:59:38.326249 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:38.326253 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:38.326308 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:38.326404 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:59:38.326998 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.327814 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.329989 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.331213 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.331329 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:38.331418 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:38.331888 (+ 470us) server_negotiation.cc:299] Negotiation successful
0620 22:59:38.332009 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260620 22:59:38.332271 10434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:38.324657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:58657 (local address 127.2.237.1:41217)
0620 22:59:38.324834 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:38.324838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:38.325297 (+ 459us) server_negotiation.cc:407] Connection header received
0620 22:59:38.325509 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:38.325517 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:38.325613 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:38.325732 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:59:38.326276 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.327270 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.329624 (+ 2354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.330744 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.330882 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:38.330981 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:38.331198 (+ 217us) server_negotiation.cc:299] Negotiation successful
0620 22:59:38.331344 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 22:59:38.333294 10069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37a9608a3fcd4226a9082d7fbf81a99c" candidate_uuid: "d3ffb93de8d841159da80de89f7171e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8154a02194f741cbb6596035cec6fcf0" is_pre_election: true
I20260620 22:59:38.333295 10356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37a9608a3fcd4226a9082d7fbf81a99c" candidate_uuid: "d3ffb93de8d841159da80de89f7171e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3846f009be844b4ab3849d5ad442e6cb" is_pre_election: true
I20260620 22:59:38.333884 10069 raft_consensus.cc:2454] T 37a9608a3fcd4226a9082d7fbf81a99c P 8154a02194f741cbb6596035cec6fcf0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3ffb93de8d841159da80de89f7171e4 in term 0.
I20260620 22:59:38.333884 10356 raft_consensus.cc:2454] T 37a9608a3fcd4226a9082d7fbf81a99c P 3846f009be844b4ab3849d5ad442e6cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3ffb93de8d841159da80de89f7171e4 in term 0.
I20260620 22:59:38.334545 10149 leader_election.cc:302] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3846f009be844b4ab3849d5ad442e6cb, d3ffb93de8d841159da80de89f7171e4; no voters:
I20260620 22:59:38.334986 10446 raft_consensus.cc:2781] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 22:59:38.335130 10446 raft_consensus.cc:482] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 22:59:38.335239 10446 raft_consensus.cc:3037] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:38.337531 10446 raft_consensus.cc:504] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8154a02194f741cbb6596035cec6fcf0" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41217 } } peers { permanent_uuid: "3846f009be844b4ab3849d5ad442e6cb" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34669 } } peers { permanent_uuid: "d3ffb93de8d841159da80de89f7171e4" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33799 } }
I20260620 22:59:38.338083 10446 leader_election.cc:288] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4 [CANDIDATE]: Term 1 election: Requested vote from peers 8154a02194f741cbb6596035cec6fcf0 (127.2.237.1:41217), 3846f009be844b4ab3849d5ad442e6cb (127.2.237.3:34669)
I20260620 22:59:38.338431 10069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37a9608a3fcd4226a9082d7fbf81a99c" candidate_uuid: "d3ffb93de8d841159da80de89f7171e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8154a02194f741cbb6596035cec6fcf0"
I20260620 22:59:38.338524 10356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37a9608a3fcd4226a9082d7fbf81a99c" candidate_uuid: "d3ffb93de8d841159da80de89f7171e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3846f009be844b4ab3849d5ad442e6cb"
I20260620 22:59:38.338639 10069 raft_consensus.cc:3037] T 37a9608a3fcd4226a9082d7fbf81a99c P 8154a02194f741cbb6596035cec6fcf0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:38.338765 10356 raft_consensus.cc:3037] T 37a9608a3fcd4226a9082d7fbf81a99c P 3846f009be844b4ab3849d5ad442e6cb [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:38.341214 10069 raft_consensus.cc:2454] T 37a9608a3fcd4226a9082d7fbf81a99c P 8154a02194f741cbb6596035cec6fcf0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3ffb93de8d841159da80de89f7171e4 in term 1.
I20260620 22:59:38.341634 10149 leader_election.cc:302] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8154a02194f741cbb6596035cec6fcf0, d3ffb93de8d841159da80de89f7171e4; no voters:
I20260620 22:59:38.341735 10356 raft_consensus.cc:2454] T 37a9608a3fcd4226a9082d7fbf81a99c P 3846f009be844b4ab3849d5ad442e6cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3ffb93de8d841159da80de89f7171e4 in term 1.
I20260620 22:59:38.341944 10446 raft_consensus.cc:2781] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:59:38.343093 10446 raft_consensus.cc:686] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4 [term 1 LEADER]: Becoming Leader. State: Replica: d3ffb93de8d841159da80de89f7171e4, State: Running, Role: LEADER
I20260620 22:59:38.343739 10446 consensus_queue.cc:237] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8154a02194f741cbb6596035cec6fcf0" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41217 } } peers { permanent_uuid: "3846f009be844b4ab3849d5ad442e6cb" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34669 } } peers { permanent_uuid: "d3ffb93de8d841159da80de89f7171e4" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33799 } }
W20260620 22:59:38.348745 10404 tablet.cc:2369] T 37a9608a3fcd4226a9082d7fbf81a99c P 3846f009be844b4ab3849d5ad442e6cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 22:59:38.350248 9149 catalog_manager.cc:5495] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3ffb93de8d841159da80de89f7171e4 (127.2.237.2). New cstate: current_term: 1 leader_uuid: "d3ffb93de8d841159da80de89f7171e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8154a02194f741cbb6596035cec6fcf0" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3846f009be844b4ab3849d5ad442e6cb" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3ffb93de8d841159da80de89f7171e4" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33799 } health_report { overall_health: HEALTHY } } }
I20260620 22:59:38.399257 10436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:38.394070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33934 (local address 127.2.237.2:33799)
0620 22:59:38.394248 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:38.394271 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:38.394288 (+ 17us) server_negotiation.cc:407] Connection header received
0620 22:59:38.394376 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:38.394379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:38.394455 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:38.394804 (+ 349us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 22:59:38.395146 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.395868 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.397194 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.397659 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.397767 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:38.398175 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 22:59:38.398817 (+ 642us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 22:59:38.398930 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:38.398994 (+ 64us) server_negotiation.cc:299] Negotiation successful
0620 22:59:38.399057 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260620 22:59:38.402436 10117 tablet.cc:2369] T 37a9608a3fcd4226a9082d7fbf81a99c P 8154a02194f741cbb6596035cec6fcf0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 22:59:38.407967 10069 raft_consensus.cc:1261] T 37a9608a3fcd4226a9082d7fbf81a99c P 8154a02194f741cbb6596035cec6fcf0 [term 1 FOLLOWER]: Refusing update from remote peer d3ffb93de8d841159da80de89f7171e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:59:38.408198 10356 raft_consensus.cc:1261] T 37a9608a3fcd4226a9082d7fbf81a99c P 3846f009be844b4ab3849d5ad442e6cb [term 1 FOLLOWER]: Refusing update from remote peer d3ffb93de8d841159da80de89f7171e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:59:38.408924 10450 consensus_queue.cc:1035] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8154a02194f741cbb6596035cec6fcf0" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:59:38.409294 10446 consensus_queue.cc:1035] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3846f009be844b4ab3849d5ad442e6cb" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:59:38.432142 10453 mvcc.cc:205] Tried to move back new op lower bound from 7299057165948686336 to 7299057165700169728. Current Snapshot: MvccSnapshot[applied={T|T < 7299057165948686336 or (T in {7299057165948686336})}]
I20260620 22:59:38.442762 9149 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 22:59:38.443008 9149 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 22:59:38.449177 9149 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 988aa53a4d75433ba11c4b68df409c91: Sending DeleteTablet for 3 replicas of tablet 37a9608a3fcd4226a9082d7fbf81a99c
I20260620 22:59:38.450601 10049 tablet_service.cc:1514] Processing DeleteTablet for tablet 37a9608a3fcd4226a9082d7fbf81a99c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:38 UTC) from {username='oryx', principal='oryx/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:47086
I20260620 22:59:38.450752 10336 tablet_service.cc:1514] Processing DeleteTablet for tablet 37a9608a3fcd4226a9082d7fbf81a99c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:38 UTC) from {username='oryx', principal='oryx/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:53120
I20260620 22:59:38.450752 10195 tablet_service.cc:1514] Processing DeleteTablet for tablet 37a9608a3fcd4226a9082d7fbf81a99c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:38 UTC) from {username='oryx', principal='oryx/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:33920
I20260620 22:59:38.466056 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 9117
I20260620 22:59:38.467644 9179 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:38.469846 10462 tablet_replica.cc:330] T 37a9608a3fcd4226a9082d7fbf81a99c P 8154a02194f741cbb6596035cec6fcf0: stopping tablet replica
I20260620 22:59:38.470366 10462 raft_consensus.cc:2229] T 37a9608a3fcd4226a9082d7fbf81a99c P 8154a02194f741cbb6596035cec6fcf0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 22:59:38.470942 10462 raft_consensus.cc:2258] T 37a9608a3fcd4226a9082d7fbf81a99c P 8154a02194f741cbb6596035cec6fcf0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:38.479236 10463 tablet_replica.cc:330] T 37a9608a3fcd4226a9082d7fbf81a99c P 3846f009be844b4ab3849d5ad442e6cb: stopping tablet replica
I20260620 22:59:38.479708 10463 raft_consensus.cc:2229] T 37a9608a3fcd4226a9082d7fbf81a99c P 3846f009be844b4ab3849d5ad442e6cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 22:59:38.480223 10463 raft_consensus.cc:2258] T 37a9608a3fcd4226a9082d7fbf81a99c P 3846f009be844b4ab3849d5ad442e6cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:38.481489 10464 tablet_replica.cc:330] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4: stopping tablet replica
I20260620 22:59:38.481907 10464 raft_consensus.cc:2229] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4 [term 1 LEADER]: Raft consensus shutting down.
I20260620 22:59:38.482637 10464 raft_consensus.cc:2258] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:38.482806 10462 ts_tablet_manager.cc:1905] T 37a9608a3fcd4226a9082d7fbf81a99c P 8154a02194f741cbb6596035cec6fcf0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:38.486019 10464 ts_tablet_manager.cc:1905] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:38.489434 10463 ts_tablet_manager.cc:1905] T 37a9608a3fcd4226a9082d7fbf81a99c P 3846f009be844b4ab3849d5ad442e6cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:38.492107 10418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:38.453904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43230 (local address 127.2.237.61:38995)
0620 22:59:38.454593 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:38.454602 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:38.464600 (+ 9998us) server_negotiation.cc:407] Connection header received
0620 22:59:38.467708 (+ 3108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:38.467718 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:38.467850 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:38.467944 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:38.470984 (+ 3040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.472055 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.474048 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.474737 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.474837 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:38.483097 (+ 8260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:38.483108 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:38.483116 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:38.483177 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:38.487837 (+ 4660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:38.488786 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:38.488795 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:38.488801 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:38.489008 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:38.489675 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:38.489683 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:38.489689 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:38.490134 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:38.490440 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:38.491255 (+ 815us) server_negotiation.cc:299] Negotiation successful
0620 22:59:38.491598 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 22:59:38.499078 10464 ts_tablet_manager.cc:1918] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 22:59:38.499248 10464 log.cc:1196] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/wal/wals/37a9608a3fcd4226a9082d7fbf81a99c
I20260620 22:59:38.499792 10464 ts_tablet_manager.cc:1939] T 37a9608a3fcd4226a9082d7fbf81a99c P d3ffb93de8d841159da80de89f7171e4: Deleting consensus metadata
I20260620 22:59:38.500765 10462 ts_tablet_manager.cc:1918] T 37a9608a3fcd4226a9082d7fbf81a99c P 8154a02194f741cbb6596035cec6fcf0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 22:59:38.500936 10462 log.cc:1196] T 37a9608a3fcd4226a9082d7fbf81a99c P 8154a02194f741cbb6596035cec6fcf0: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/wal/wals/37a9608a3fcd4226a9082d7fbf81a99c
I20260620 22:59:38.505584 10463 ts_tablet_manager.cc:1918] T 37a9608a3fcd4226a9082d7fbf81a99c P 3846f009be844b4ab3849d5ad442e6cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 22:59:38.505766 10463 log.cc:1196] T 37a9608a3fcd4226a9082d7fbf81a99c P 3846f009be844b4ab3849d5ad442e6cb: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/wal/wals/37a9608a3fcd4226a9082d7fbf81a99c
I20260620 22:59:38.506790 10463 ts_tablet_manager.cc:1939] T 37a9608a3fcd4226a9082d7fbf81a99c P 3846f009be844b4ab3849d5ad442e6cb: Deleting consensus metadata
I20260620 22:59:38.507861 10462 ts_tablet_manager.cc:1939] T 37a9608a3fcd4226a9082d7fbf81a99c P 8154a02194f741cbb6596035cec6fcf0: Deleting consensus metadata
I20260620 22:59:38.514341 10416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:38.465164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42756 (local address 127.2.237.60:42911)
0620 22:59:38.465365 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:38.465370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:38.493659 (+ 28289us) server_negotiation.cc:407] Connection header received
0620 22:59:38.493985 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:38.493994 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:38.494080 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:38.494167 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:38.496722 (+ 2555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.498141 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.501330 (+ 3189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.502266 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.502392 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:38.507720 (+ 5328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:38.507731 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:38.507734 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:38.507766 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:38.511226 (+ 3460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:38.512023 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:38.512029 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:38.512033 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:38.512190 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:38.512631 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:38.512636 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:38.512640 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:38.512969 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:38.513201 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:38.513746 (+ 545us) server_negotiation.cc:299] Negotiation successful
0620 22:59:38.513992 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260620 22:59:38.627592 9134 catalog_manager.cc:4841] TS 8154a02194f741cbb6596035cec6fcf0 (127.2.237.1:41217): tablet 37a9608a3fcd4226a9082d7fbf81a99c (table test-table [id=5a7b09a07f014288b2959f2d598e3b0e]) successfully deleted
I20260620 22:59:38.627729 9135 catalog_manager.cc:4841] TS d3ffb93de8d841159da80de89f7171e4 (127.2.237.2:33799): tablet 37a9608a3fcd4226a9082d7fbf81a99c (table test-table [id=5a7b09a07f014288b2959f2d598e3b0e]) successfully deleted
I20260620 22:59:38.627986 9134 catalog_manager.cc:4841] TS 3846f009be844b4ab3849d5ad442e6cb (127.2.237.3:34669): tablet 37a9608a3fcd4226a9082d7fbf81a99c (table test-table [id=5a7b09a07f014288b2959f2d598e3b0e]) successfully deleted
I20260620 22:59:38.628363 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 9117
I20260620 22:59:38.675246 10416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:38.656020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42758 (local address 127.2.237.60:42911)
0620 22:59:38.656221 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:38.656227 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:38.657197 (+ 970us) server_negotiation.cc:407] Connection header received
0620 22:59:38.657465 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:38.657471 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:38.657552 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:38.657626 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:38.660224 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.661339 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.663283 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.663954 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.664054 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:38.669362 (+ 5308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:38.669368 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:38.669373 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:38.669413 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:38.672431 (+ 3018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:38.673275 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:38.673281 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:38.673285 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:38.673427 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:38.673860 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:38.673867 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:38.673870 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:38.674167 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:38.674400 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:38.674715 (+ 315us) server_negotiation.cc:299] Negotiation successful
0620 22:59:38.674926 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 22:59:38.683095 10418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:38.655526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43234 (local address 127.2.237.61:38995)
0620 22:59:38.655718 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:38.655726 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:38.669711 (+ 13985us) server_negotiation.cc:407] Connection header received
0620 22:59:38.669844 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:38.669850 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:38.669939 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:38.670046 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:38.671303 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.672399 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.673955 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:38.674658 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:38.674788 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:38.678011 (+ 3223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:38.678016 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:38.678019 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:38.678048 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:38.680611 (+ 2563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:38.681318 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:38.681321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:38.681323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:38.681449 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:38.681819 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:38.681822 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:38.681824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:38.682057 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:38.682238 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:38.682648 (+ 410us) server_negotiation.cc:299] Negotiation successful
0620 22:59:38.682802 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 22:59:39.436403 10474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:39.435673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.2.237.2:34221)
0620 22:59:39.436090 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:39.436193 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":201,"threads_started":1}
W20260620 22:59:39.436882 10260 heartbeater.cc:644] Failed to heartbeat to 127.2.237.62:36895 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
I20260620 22:59:39.438092 10475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:39.437191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.2.237.3:39915)
0620 22:59:39.437679 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:39.437839 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":256,"threads_started":1}
W20260620 22:59:39.438472 10401 heartbeater.cc:644] Failed to heartbeat to 127.2.237.62:36895 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
I20260620 22:59:39.440063 10476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:39.438766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.2.237.1:46981)
0620 22:59:39.439772 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:39.439888 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":882,"thread_start_us":789,"threads_started":1}
W20260620 22:59:39.440466 10114 heartbeater.cc:644] Failed to heartbeat to 127.2.237.62:36895 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
I20260620 22:59:39.949671 10477 raft_consensus.cc:482] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 988aa53a4d75433ba11c4b68df409c91)
I20260620 22:59:39.949762 10478 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 988aa53a4d75433ba11c4b68df409c91)
I20260620 22:59:39.949976 10477 raft_consensus.cc:504] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: false } }
I20260620 22:59:39.950045 10478 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: false } }
I20260620 22:59:39.951464 10478 leader_election.cc:288] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 988aa53a4d75433ba11c4b68df409c91 (127.2.237.62:36895), c582ab7db2fb468aaa0862be09f6e534 (127.2.237.60:42911)
I20260620 22:59:39.957712 10477 leader_election.cc:288] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 988aa53a4d75433ba11c4b68df409c91 (127.2.237.62:36895), 65dcb1d2184d4d41bb04de52e7e5e2aa (127.2.237.61:38995)
I20260620 22:59:39.960630 10479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:39.951611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.0.0.1:57350)
0620 22:59:39.960084 (+ 8473us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:39.960298 (+ 214us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8282,"threads_started":1}
W20260620 22:59:39.961629 9921 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
I20260620 22:59:39.963888 10482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:39.951604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.0.0.1:57338)
0620 22:59:39.963334 (+ 11730us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:39.963553 (+ 219us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11553,"threads_started":1}
W20260620 22:59:39.966647 9733 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
I20260620 22:59:39.967650 10481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:39.966393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.0.0.1:57382)
0620 22:59:39.967220 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:39.967391 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":658}
W20260620 22:59:39.967890 9921 leader_election.cc:334] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 988aa53a4d75433ba11c4b68df409c91 (127.2.237.62:36895): Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
I20260620 22:59:39.968888 10480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:39.968378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.0.0.1:57398)
0620 22:59:39.968566 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:39.968725 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260620 22:59:39.969135 9733 leader_election.cc:334] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 988aa53a4d75433ba11c4b68df409c91 (127.2.237.62:36895): Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
I20260620 22:59:39.972726 10482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:39.952025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:42911 (local address 127.0.0.1:42764)
0620 22:59:39.964428 (+ 12403us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:39.964452 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:39.964608 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:39.966784 (+ 2176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:39.966789 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:39.966835 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:59:39.967125 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:39.967139 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:39.969093 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:39.969099 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:39.970443 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:39.970454 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:39.971896 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:39.971902 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:39.972115 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:39.972148 (+ 33us) client_negotiation.cc:769] Sending connection context
0620 22:59:39.972246 (+ 98us) client_negotiation.cc:240] Negotiation successful
0620 22:59:39.972380 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12281,"mutex_wait_us":41,"spinlock_wait_cycles":1408,"thread_start_us":12027,"threads_started":1}
I20260620 22:59:39.972743 10479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:39.951643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:38995 (local address 127.0.0.1:43242)
0620 22:59:39.960870 (+ 9227us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:39.960894 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:39.961079 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:39.967012 (+ 5933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:39.967017 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:39.967046 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:59:39.967349 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:39.967367 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:39.969048 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:39.969052 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:39.970317 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:39.970328 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:39.971899 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:39.971904 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:39.972115 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:39.972148 (+ 33us) client_negotiation.cc:769] Sending connection context
0620 22:59:39.972246 (+ 98us) client_negotiation.cc:240] Negotiation successful
0620 22:59:39.972380 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9115,"mutex_wait_us":9,"thread_start_us":12322,"threads_started":1}
I20260620 22:59:39.972952 10485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:39.965174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43242 (local address 127.2.237.61:38995)
0620 22:59:39.966678 (+ 1504us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:39.966684 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:39.966700 (+ 16us) server_negotiation.cc:407] Connection header received
0620 22:59:39.966756 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:39.966761 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:39.966824 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:39.966918 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:59:39.967497 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:39.968920 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:39.970466 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:39.971719 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:39.971849 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:39.971988 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:39.972283 (+ 295us) server_negotiation.cc:299] Negotiation successful
0620 22:59:39.972412 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":1328,"thread_start_us":1154,"threads_started":1}
I20260620 22:59:39.973039 10484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:39.964285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42764 (local address 127.2.237.60:42911)
0620 22:59:39.966407 (+ 2122us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:39.966413 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:39.966435 (+ 22us) server_negotiation.cc:407] Connection header received
0620 22:59:39.966493 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:39.966498 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:39.966595 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:39.967041 (+ 446us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:59:39.967570 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:39.968860 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:39.970605 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:39.971735 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:39.971849 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:39.971995 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:39.972283 (+ 288us) server_negotiation.cc:299] Negotiation successful
0620 22:59:39.972424 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":261,"server-negotiator.queue_time_us":1942,"thread_start_us":1791,"threads_started":1}
I20260620 22:59:39.973554 9945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c582ab7db2fb468aaa0862be09f6e534" is_pre_election: true
I20260620 22:59:39.973675 9756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c582ab7db2fb468aaa0862be09f6e534" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" is_pre_election: true
I20260620 22:59:39.973869 9945 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65dcb1d2184d4d41bb04de52e7e5e2aa in term 1.
I20260620 22:59:39.974023 9756 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c582ab7db2fb468aaa0862be09f6e534 in term 1.
I20260620 22:59:39.974370 9731 leader_election.cc:302] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 65dcb1d2184d4d41bb04de52e7e5e2aa, c582ab7db2fb468aaa0862be09f6e534; no voters: 988aa53a4d75433ba11c4b68df409c91
I20260620 22:59:39.974488 9920 leader_election.cc:302] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 65dcb1d2184d4d41bb04de52e7e5e2aa, c582ab7db2fb468aaa0862be09f6e534; no voters: 988aa53a4d75433ba11c4b68df409c91
I20260620 22:59:39.974805 10478 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260620 22:59:39.974844 10477 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260620 22:59:39.974943 10478 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 1 FOLLOWER]: Starting leader election (detected failure of leader 988aa53a4d75433ba11c4b68df409c91)
I20260620 22:59:39.974958 10477 raft_consensus.cc:482] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 988aa53a4d75433ba11c4b68df409c91)
I20260620 22:59:39.975047 10477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [term 1 FOLLOWER]: Advancing to term 2
I20260620 22:59:39.975057 10478 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 1 FOLLOWER]: Advancing to term 2
I20260620 22:59:39.978083 10477 raft_consensus.cc:504] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: false } }
I20260620 22:59:39.978099 10478 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: false } }
I20260620 22:59:39.978828 10478 leader_election.cc:288] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [CANDIDATE]: Term 2 election: Requested vote from peers 988aa53a4d75433ba11c4b68df409c91 (127.2.237.62:36895), c582ab7db2fb468aaa0862be09f6e534 (127.2.237.60:42911)
I20260620 22:59:39.978883 10477 leader_election.cc:288] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [CANDIDATE]: Term 2 election: Requested vote from peers 988aa53a4d75433ba11c4b68df409c91 (127.2.237.62:36895), 65dcb1d2184d4d41bb04de52e7e5e2aa (127.2.237.61:38995)
I20260620 22:59:39.979475 10482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:39.979117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.0.0.1:57406)
0620 22:59:39.979258 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:39.979356 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260620 22:59:39.979486 10479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:39.978962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.0.0.1:57422)
0620 22:59:39.979193 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:39.979305 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260620 22:59:39.979413 9945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c582ab7db2fb468aaa0862be09f6e534"
I20260620 22:59:39.979772 9945 raft_consensus.cc:2379] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 65dcb1d2184d4d41bb04de52e7e5e2aa in current term 2: Already voted for candidate c582ab7db2fb468aaa0862be09f6e534 in this term.
I20260620 22:59:39.980269 9756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c582ab7db2fb468aaa0862be09f6e534" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa"
I20260620 22:59:39.980547 10479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:39.980087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.0.0.1:57436)
0620 22:59:39.980303 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:39.980408 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260620 22:59:39.980571 9756 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c582ab7db2fb468aaa0862be09f6e534 in current term 2: Already voted for candidate 65dcb1d2184d4d41bb04de52e7e5e2aa in this term.
W20260620 22:59:39.980739 9921 leader_election.cc:334] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 988aa53a4d75433ba11c4b68df409c91 (127.2.237.62:36895): Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
I20260620 22:59:39.981062 10482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:39.980699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.0.0.1:57450)
0620 22:59:39.980846 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:39.980926 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260620 22:59:39.981040 9920 leader_election.cc:302] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c582ab7db2fb468aaa0862be09f6e534; no voters: 65dcb1d2184d4d41bb04de52e7e5e2aa, 988aa53a4d75433ba11c4b68df409c91
I20260620 22:59:39.981299 10477 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260620 22:59:39.981464 9733 leader_election.cc:334] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 988aa53a4d75433ba11c4b68df409c91 (127.2.237.62:36895): Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
I20260620 22:59:39.981596 9733 leader_election.cc:302] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 65dcb1d2184d4d41bb04de52e7e5e2aa; no voters: 988aa53a4d75433ba11c4b68df409c91, c582ab7db2fb468aaa0862be09f6e534
I20260620 22:59:39.981806 10478 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260620 22:59:40.438169 10487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:40.437525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.2.237.2:40591)
0620 22:59:40.437936 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:40.438016 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":223,"threads_started":1}
I20260620 22:59:40.439873 10488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:40.439133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.2.237.3:33083)
0620 22:59:40.439541 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:40.439630 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":194,"threads_started":1}
I20260620 22:59:40.442047 10489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:40.441339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.2.237.1:50409)
0620 22:59:40.441769 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:40.441858 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":232,"threads_started":1}
I20260620 22:59:41.439702 10490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:41.438949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.2.237.2:55755)
0620 22:59:41.439396 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:41.439475 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":235,"threads_started":1}
I20260620 22:59:41.442483 10491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:41.441407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.2.237.3:41721)
0620 22:59:41.442125 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:41.442234 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":551,"threads_started":1}
I20260620 22:59:41.443809 10492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:41.442921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.2.237.1:45909)
0620 22:59:41.443545 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:41.443662 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":401,"threads_started":1}
I20260620 22:59:41.641726 10493 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 22:59:41.641983 10493 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: false } }
I20260620 22:59:41.642781 10493 leader_election.cc:288] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 988aa53a4d75433ba11c4b68df409c91 (127.2.237.62:36895), c582ab7db2fb468aaa0862be09f6e534 (127.2.237.60:42911)
I20260620 22:59:41.643344 9945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c582ab7db2fb468aaa0862be09f6e534" is_pre_election: true
I20260620 22:59:41.643616 9945 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65dcb1d2184d4d41bb04de52e7e5e2aa in term 2.
I20260620 22:59:41.643718 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:41.642918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.0.0.1:57682)
0620 22:59:41.643408 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:41.643501 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":259,"threads_started":1}
I20260620 22:59:41.644145 9731 leader_election.cc:302] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65dcb1d2184d4d41bb04de52e7e5e2aa, c582ab7db2fb468aaa0862be09f6e534; no voters:
I20260620 22:59:41.644426 10493 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260620 22:59:41.644573 10493 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 22:59:41.644688 10493 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 2 FOLLOWER]: Advancing to term 3
I20260620 22:59:41.645126 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:41.644813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.0.0.1:57688)
0620 22:59:41.644939 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:41.645019 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260620 22:59:41.645292 9733 leader_election.cc:334] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 988aa53a4d75433ba11c4b68df409c91 (127.2.237.62:36895): Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
I20260620 22:59:41.647114 10493 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: false } }
I20260620 22:59:41.647663 10493 leader_election.cc:288] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [CANDIDATE]: Term 3 election: Requested vote from peers 988aa53a4d75433ba11c4b68df409c91 (127.2.237.62:36895), c582ab7db2fb468aaa0862be09f6e534 (127.2.237.60:42911)
I20260620 22:59:41.648233 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:41.647775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.0.0.1:57704)
0620 22:59:41.647954 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:41.648049 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260620 22:59:41.648156 9945 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c582ab7db2fb468aaa0862be09f6e534"
I20260620 22:59:41.648381 9945 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [term 2 FOLLOWER]: Advancing to term 3
I20260620 22:59:41.649255 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:41.648865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.0.0.1:57710)
0620 22:59:41.649014 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:41.649122 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260620 22:59:41.649464 9733 leader_election.cc:334] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 988aa53a4d75433ba11c4b68df409c91 (127.2.237.62:36895): Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
I20260620 22:59:41.650780 9945 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65dcb1d2184d4d41bb04de52e7e5e2aa in term 3.
I20260620 22:59:41.651293 9731 leader_election.cc:302] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 65dcb1d2184d4d41bb04de52e7e5e2aa, c582ab7db2fb468aaa0862be09f6e534; no voters: 988aa53a4d75433ba11c4b68df409c91
I20260620 22:59:41.651643 10493 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 3 FOLLOWER]: Leader election won for term 3
I20260620 22:59:41.652060 10493 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [term 3 LEADER]: Becoming Leader. State: Replica: 65dcb1d2184d4d41bb04de52e7e5e2aa, State: Running, Role: LEADER
I20260620 22:59:41.652462 10493 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: false } }
I20260620 22:59:41.654459 10496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65dcb1d2184d4d41bb04de52e7e5e2aa. Latest consensus state: current_term: 3 leader_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: false } } }
I20260620 22:59:41.654834 10496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:41.655494 10498 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 22:59:41.659179 10498 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 22:59:41.659982 10498 catalog_manager.cc:1259] Loaded cluster ID: 77a818f56f2d4bb9a22550a45993c572
I20260620 22:59:41.660059 10498 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 22:59:41.661204 10498 catalog_manager.cc:1504] Loading token signing keys...
I20260620 22:59:41.662007 10498 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa: Loaded TSK: 0
I20260620 22:59:41.662918 10498 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 22:59:41.718933 9747 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43234:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 22:59:41.729135 9747 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 22:59:41.733898 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:41.733458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.0.0.1:57724)
0620 22:59:41.733632 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:41.733727 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260620 22:59:41.733992 9945 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [term 3 FOLLOWER]: Refusing update from remote peer 65dcb1d2184d4d41bb04de52e7e5e2aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260620 22:59:41.734738 10496 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260620 22:59:41.734982 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:41.734564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.0.0.1:57730)
0620 22:59:41.734776 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:41.734855 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260620 22:59:41.735175 9733 consensus_peers.cc:487] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa -> Peer 988aa53a4d75433ba11c4b68df409c91 (127.2.237.62:36895): Couldn't send request to peer 988aa53a4d75433ba11c4b68df409c91. Status: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260620 22:59:41.737660 10500 sys_catalog.cc:455] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65dcb1d2184d4d41bb04de52e7e5e2aa. Latest consensus state: current_term: 3 leader_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: false } } }
I20260620 22:59:41.738013 10500 sys_catalog.cc:458] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 22:59:41.739601 10493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: false } } }
I20260620 22:59:41.739859 10493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:41.740232 10500 sys_catalog.cc:455] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "988aa53a4d75433ba11c4b68df409c91" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 36895 } } peers { permanent_uuid: "65dcb1d2184d4d41bb04de52e7e5e2aa" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 38995 } attrs { promote: false } } peers { permanent_uuid: "c582ab7db2fb468aaa0862be09f6e534" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 42911 } attrs { promote: false } } }
I20260620 22:59:41.740491 10500 sys_catalog.cc:458] T 00000000000000000000000000000000 P c582ab7db2fb468aaa0862be09f6e534 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 22:59:41.755182 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:41.747189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:34669 (local address 127.0.0.1:53134)
0620 22:59:41.747341 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:41.747361 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:41.747471 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:41.748616 (+ 1145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:41.748623 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:41.748655 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:59:41.748975 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:41.748991 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:41.750460 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:41.750465 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:41.752141 (+ 1676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:41.752153 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:41.754340 (+ 2187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:41.754347 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:41.754535 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:41.754628 (+ 93us) client_negotiation.cc:769] Sending connection context
0620 22:59:41.754723 (+ 95us) client_negotiation.cc:240] Negotiation successful
0620 22:59:41.754876 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260620 22:59:41.755381 10505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:41.747304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53134 (local address 127.2.237.3:34669)
0620 22:59:41.748255 (+ 951us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:41.748261 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:41.748280 (+ 19us) server_negotiation.cc:407] Connection header received
0620 22:59:41.748347 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:41.748352 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:41.748419 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:41.749126 (+ 707us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:59:41.749290 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:41.750321 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:41.752361 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:41.754168 (+ 1807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:41.754331 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:41.754486 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:41.754758 (+ 272us) server_negotiation.cc:299] Negotiation successful
0620 22:59:41.754952 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":816,"thread_start_us":730,"threads_started":1}
I20260620 22:59:41.756439 10336 tablet_service.cc:1467] Processing CreateTablet for tablet fa4f52b3bc5e49358119e078c411af7d (DEFAULT_TABLE table=test-table [id=06015362d9d1410882e7b7324220b73f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:59:41.756995 10336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa4f52b3bc5e49358119e078c411af7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:41.757354 10506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:41.747581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:41217 (local address 127.0.0.1:47092)
0620 22:59:41.748146 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:41.748169 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:41.748283 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:41.751340 (+ 3057us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:41.751348 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:41.751390 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:59:41.752013 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:41.752031 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:41.753338 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:41.753344 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:41.755207 (+ 1863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:41.755222 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:41.756566 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:41.756572 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:41.756770 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:41.756800 (+ 30us) client_negotiation.cc:769] Sending connection context
0620 22:59:41.756886 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 22:59:41.757026 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":1221,"threads_started":1}
I20260620 22:59:41.758023 10507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:41.747932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47092 (local address 127.2.237.1:41217)
0620 22:59:41.750973 (+ 3041us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:41.750979 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:41.750996 (+ 17us) server_negotiation.cc:407] Connection header received
0620 22:59:41.751072 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:41.751077 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:41.751161 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:41.751259 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:59:41.752240 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:41.753185 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:41.755415 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:41.756383 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:41.756521 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:41.756622 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:41.757647 (+ 1025us) server_negotiation.cc:299] Negotiation successful
0620 22:59:41.757762 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2912,"thread_start_us":2811,"threads_started":1}
I20260620 22:59:41.759469 10049 tablet_service.cc:1467] Processing CreateTablet for tablet fa4f52b3bc5e49358119e078c411af7d (DEFAULT_TABLE table=test-table [id=06015362d9d1410882e7b7324220b73f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:59:41.760017 10049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa4f52b3bc5e49358119e078c411af7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:41.766744 10510 tablet_bootstrap.cc:492] T fa4f52b3bc5e49358119e078c411af7d P 3846f009be844b4ab3849d5ad442e6cb: Bootstrap starting.
I20260620 22:59:41.768105 10511 tablet_bootstrap.cc:492] T fa4f52b3bc5e49358119e078c411af7d P 8154a02194f741cbb6596035cec6fcf0: Bootstrap starting.
I20260620 22:59:41.768613 10510 tablet_bootstrap.cc:654] T fa4f52b3bc5e49358119e078c411af7d P 3846f009be844b4ab3849d5ad442e6cb: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:41.769802 10511 tablet_bootstrap.cc:654] T fa4f52b3bc5e49358119e078c411af7d P 8154a02194f741cbb6596035cec6fcf0: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:41.771554 10511 tablet_bootstrap.cc:492] T fa4f52b3bc5e49358119e078c411af7d P 8154a02194f741cbb6596035cec6fcf0: No bootstrap required, opened a new log
I20260620 22:59:41.771544 10510 tablet_bootstrap.cc:492] T fa4f52b3bc5e49358119e078c411af7d P 3846f009be844b4ab3849d5ad442e6cb: No bootstrap required, opened a new log
I20260620 22:59:41.771586 10508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:41.748303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:33799 (local address 127.0.0.1:33948)
0620 22:59:41.748947 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:41.748968 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:41.749064 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:41.764430 (+ 15366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:41.764437 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:41.764465 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:59:41.764766 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:41.764778 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:41.766158 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:41.766165 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:41.769441 (+ 3276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:41.769456 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:41.770779 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:41.770785 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:41.770978 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:41.771011 (+ 33us) client_negotiation.cc:769] Sending connection context
0620 22:59:41.771107 (+ 96us) client_negotiation.cc:240] Negotiation successful
0620 22:59:41.771240 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":354,"threads_started":1}
I20260620 22:59:41.771704 10510 ts_tablet_manager.cc:1397] T fa4f52b3bc5e49358119e078c411af7d P 3846f009be844b4ab3849d5ad442e6cb: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260620 22:59:41.771713 10511 ts_tablet_manager.cc:1397] T fa4f52b3bc5e49358119e078c411af7d P 8154a02194f741cbb6596035cec6fcf0: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260620 22:59:41.772356 10509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:41.748415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33948 (local address 127.2.237.2:33799)
0620 22:59:41.764070 (+ 15655us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:41.764077 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:41.764099 (+ 22us) server_negotiation.cc:407] Connection header received
0620 22:59:41.764186 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:41.764192 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:41.764270 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:41.764394 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:59:41.764957 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:41.766018 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:41.769587 (+ 3569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:41.770522 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:41.770721 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:41.770831 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:41.771904 (+ 1073us) server_negotiation.cc:299] Negotiation successful
0620 22:59:41.772060 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15531,"thread_start_us":16116,"threads_started":1}
I20260620 22:59:41.772312 10511 raft_consensus.cc:348] T fa4f52b3bc5e49358119e078c411af7d P 8154a02194f741cbb6596035cec6fcf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3846f009be844b4ab3849d5ad442e6cb" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34669 } } peers { permanent_uuid: "8154a02194f741cbb6596035cec6fcf0" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41217 } } peers { permanent_uuid: "d3ffb93de8d841159da80de89f7171e4" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33799 } }
I20260620 22:59:41.772564 10511 raft_consensus.cc:374] T fa4f52b3bc5e49358119e078c411af7d P 8154a02194f741cbb6596035cec6fcf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:41.772675 10511 raft_consensus.cc:729] T fa4f52b3bc5e49358119e078c411af7d P 8154a02194f741cbb6596035cec6fcf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8154a02194f741cbb6596035cec6fcf0, State: Initialized, Role: FOLLOWER
I20260620 22:59:41.772895 10510 raft_consensus.cc:348] T fa4f52b3bc5e49358119e078c411af7d P 3846f009be844b4ab3849d5ad442e6cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3846f009be844b4ab3849d5ad442e6cb" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34669 } } peers { permanent_uuid: "8154a02194f741cbb6596035cec6fcf0" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41217 } } peers { permanent_uuid: "d3ffb93de8d841159da80de89f7171e4" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33799 } }
I20260620 22:59:41.772953 10511 consensus_queue.cc:260] T fa4f52b3bc5e49358119e078c411af7d P 8154a02194f741cbb6596035cec6fcf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3846f009be844b4ab3849d5ad442e6cb" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34669 } } peers { permanent_uuid: "8154a02194f741cbb6596035cec6fcf0" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41217 } } peers { permanent_uuid: "d3ffb93de8d841159da80de89f7171e4" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33799 } }
I20260620 22:59:41.773106 10510 raft_consensus.cc:374] T fa4f52b3bc5e49358119e078c411af7d P 3846f009be844b4ab3849d5ad442e6cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:41.773120 10195 tablet_service.cc:1467] Processing CreateTablet for tablet fa4f52b3bc5e49358119e078c411af7d (DEFAULT_TABLE table=test-table [id=06015362d9d1410882e7b7324220b73f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:59:41.773197 10510 raft_consensus.cc:729] T fa4f52b3bc5e49358119e078c411af7d P 3846f009be844b4ab3849d5ad442e6cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3846f009be844b4ab3849d5ad442e6cb, State: Initialized, Role: FOLLOWER
I20260620 22:59:41.773433 10510 consensus_queue.cc:260] T fa4f52b3bc5e49358119e078c411af7d P 3846f009be844b4ab3849d5ad442e6cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3846f009be844b4ab3849d5ad442e6cb" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34669 } } peers { permanent_uuid: "8154a02194f741cbb6596035cec6fcf0" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41217 } } peers { permanent_uuid: "d3ffb93de8d841159da80de89f7171e4" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33799 } }
I20260620 22:59:41.773612 10195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa4f52b3bc5e49358119e078c411af7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:41.774766 10510 ts_tablet_manager.cc:1428] T fa4f52b3bc5e49358119e078c411af7d P 3846f009be844b4ab3849d5ad442e6cb: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260620 22:59:41.779376 10491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:41.774435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.2.237.3:41689)
0620 22:59:41.779095 (+ 4660us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:41.779212 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4524}
I20260620 22:59:41.780499 10491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:41.780064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.2.237.3:39653)
0620 22:59:41.780289 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:41.780360 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
I20260620 22:59:41.782040 10492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:41.781581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.2.237.1:43357)
0620 22:59:41.781795 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:41.781879 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260620 22:59:41.782788 10511 ts_tablet_manager.cc:1428] T fa4f52b3bc5e49358119e078c411af7d P 8154a02194f741cbb6596035cec6fcf0: Time spent starting tablet: real 0.011s user 0.005s sys 0.004s
I20260620 22:59:41.783826 10116 heartbeater.cc:503] Master 127.2.237.61:38995 was elected leader, sending a full tablet report...
I20260620 22:59:41.783877 10492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:41.783421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.2.237.1:59087)
0620 22:59:41.783629 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:41.783700 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260620 22:59:41.787037 10516 tablet_bootstrap.cc:492] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4: Bootstrap starting.
I20260620 22:59:41.788745 10516 tablet_bootstrap.cc:654] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:41.791343 10516 tablet_bootstrap.cc:492] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4: No bootstrap required, opened a new log
I20260620 22:59:41.791496 10516 ts_tablet_manager.cc:1397] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260620 22:59:41.792098 10516 raft_consensus.cc:348] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3846f009be844b4ab3849d5ad442e6cb" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34669 } } peers { permanent_uuid: "8154a02194f741cbb6596035cec6fcf0" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41217 } } peers { permanent_uuid: "d3ffb93de8d841159da80de89f7171e4" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33799 } }
I20260620 22:59:41.792306 10516 raft_consensus.cc:374] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:41.792411 10516 raft_consensus.cc:729] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3ffb93de8d841159da80de89f7171e4, State: Initialized, Role: FOLLOWER
I20260620 22:59:41.792636 10516 consensus_queue.cc:260] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3846f009be844b4ab3849d5ad442e6cb" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34669 } } peers { permanent_uuid: "8154a02194f741cbb6596035cec6fcf0" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41217 } } peers { permanent_uuid: "d3ffb93de8d841159da80de89f7171e4" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33799 } }
I20260620 22:59:41.794067 10490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:41.793684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.2.237.2:35771)
0620 22:59:41.793831 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:41.793919 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260620 22:59:41.794409 10516 ts_tablet_manager.cc:1428] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260620 22:59:41.795240 10262 heartbeater.cc:503] Master 127.2.237.61:38995 was elected leader, sending a full tablet report...
I20260620 22:59:41.796133 10490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:41.795706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.2.237.2:36863)
0620 22:59:41.795900 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:41.796003 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260620 22:59:41.970634 10518 raft_consensus.cc:482] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 22:59:41.970871 10518 raft_consensus.cc:504] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3846f009be844b4ab3849d5ad442e6cb" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34669 } } peers { permanent_uuid: "8154a02194f741cbb6596035cec6fcf0" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41217 } } peers { permanent_uuid: "d3ffb93de8d841159da80de89f7171e4" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33799 } }
I20260620 22:59:41.971511 10518 leader_election.cc:288] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3846f009be844b4ab3849d5ad442e6cb (127.2.237.3:34669), 8154a02194f741cbb6596035cec6fcf0 (127.2.237.1:41217)
I20260620 22:59:41.972187 10356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa4f52b3bc5e49358119e078c411af7d" candidate_uuid: "d3ffb93de8d841159da80de89f7171e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3846f009be844b4ab3849d5ad442e6cb" is_pre_election: true
I20260620 22:59:41.972265 10069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa4f52b3bc5e49358119e078c411af7d" candidate_uuid: "d3ffb93de8d841159da80de89f7171e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8154a02194f741cbb6596035cec6fcf0" is_pre_election: true
I20260620 22:59:41.972419 10356 raft_consensus.cc:2454] T fa4f52b3bc5e49358119e078c411af7d P 3846f009be844b4ab3849d5ad442e6cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3ffb93de8d841159da80de89f7171e4 in term 0.
I20260620 22:59:41.972543 10069 raft_consensus.cc:2454] T fa4f52b3bc5e49358119e078c411af7d P 8154a02194f741cbb6596035cec6fcf0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3ffb93de8d841159da80de89f7171e4 in term 0.
I20260620 22:59:41.972916 10149 leader_election.cc:302] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3846f009be844b4ab3849d5ad442e6cb, d3ffb93de8d841159da80de89f7171e4; no voters:
I20260620 22:59:41.973161 10518 raft_consensus.cc:2781] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 22:59:41.973277 10518 raft_consensus.cc:482] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 22:59:41.973368 10518 raft_consensus.cc:3037] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:41.975445 10518 raft_consensus.cc:504] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3846f009be844b4ab3849d5ad442e6cb" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34669 } } peers { permanent_uuid: "8154a02194f741cbb6596035cec6fcf0" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41217 } } peers { permanent_uuid: "d3ffb93de8d841159da80de89f7171e4" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33799 } }
I20260620 22:59:41.975937 10518 leader_election.cc:288] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4 [CANDIDATE]: Term 1 election: Requested vote from peers 3846f009be844b4ab3849d5ad442e6cb (127.2.237.3:34669), 8154a02194f741cbb6596035cec6fcf0 (127.2.237.1:41217)
I20260620 22:59:41.976413 10356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa4f52b3bc5e49358119e078c411af7d" candidate_uuid: "d3ffb93de8d841159da80de89f7171e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3846f009be844b4ab3849d5ad442e6cb"
I20260620 22:59:41.976452 10069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa4f52b3bc5e49358119e078c411af7d" candidate_uuid: "d3ffb93de8d841159da80de89f7171e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8154a02194f741cbb6596035cec6fcf0"
I20260620 22:59:41.976632 10356 raft_consensus.cc:3037] T fa4f52b3bc5e49358119e078c411af7d P 3846f009be844b4ab3849d5ad442e6cb [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:41.976631 10069 raft_consensus.cc:3037] T fa4f52b3bc5e49358119e078c411af7d P 8154a02194f741cbb6596035cec6fcf0 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:41.979626 10356 raft_consensus.cc:2454] T fa4f52b3bc5e49358119e078c411af7d P 3846f009be844b4ab3849d5ad442e6cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3ffb93de8d841159da80de89f7171e4 in term 1.
I20260620 22:59:41.979626 10069 raft_consensus.cc:2454] T fa4f52b3bc5e49358119e078c411af7d P 8154a02194f741cbb6596035cec6fcf0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3ffb93de8d841159da80de89f7171e4 in term 1.
I20260620 22:59:41.980020 10149 leader_election.cc:302] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8154a02194f741cbb6596035cec6fcf0, d3ffb93de8d841159da80de89f7171e4; no voters:
I20260620 22:59:41.980322 10518 raft_consensus.cc:2781] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:59:41.980790 10518 raft_consensus.cc:686] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4 [term 1 LEADER]: Becoming Leader. State: Replica: d3ffb93de8d841159da80de89f7171e4, State: Running, Role: LEADER
I20260620 22:59:41.981196 10518 consensus_queue.cc:237] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3846f009be844b4ab3849d5ad442e6cb" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34669 } } peers { permanent_uuid: "8154a02194f741cbb6596035cec6fcf0" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41217 } } peers { permanent_uuid: "d3ffb93de8d841159da80de89f7171e4" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33799 } }
I20260620 22:59:41.981906 10490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:41.981408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.2.237.2:45141)
0620 22:59:41.981658 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:41.981752 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
I20260620 22:59:41.984157 9746 catalog_manager.cc:5495] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to d3ffb93de8d841159da80de89f7171e4 (127.2.237.2). New cstate: current_term: 1 leader_uuid: "d3ffb93de8d841159da80de89f7171e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3846f009be844b4ab3849d5ad442e6cb" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8154a02194f741cbb6596035cec6fcf0" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3ffb93de8d841159da80de89f7171e4" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 33799 } health_report { overall_health: HEALTHY } } }
W20260620 22:59:41.996527 10263 tablet.cc:2369] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 22:59:42.052848 10509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:42.048391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33962 (local address 127.2.237.2:33799)
0620 22:59:42.048580 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:42.048584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:42.048601 (+ 17us) server_negotiation.cc:407] Connection header received
0620 22:59:42.048679 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:42.048682 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:42.048744 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:42.048826 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 22:59:42.049120 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:42.049860 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:42.051262 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:42.051722 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:42.051817 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:42.052222 (+ 405us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 22:59:42.052325 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 22:59:42.052390 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:42.052593 (+ 203us) server_negotiation.cc:299] Negotiation successful
0620 22:59:42.052654 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 22:59:42.055979 10069 raft_consensus.cc:1261] T fa4f52b3bc5e49358119e078c411af7d P 8154a02194f741cbb6596035cec6fcf0 [term 1 FOLLOWER]: Refusing update from remote peer d3ffb93de8d841159da80de89f7171e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:59:42.056036 10356 raft_consensus.cc:1261] T fa4f52b3bc5e49358119e078c411af7d P 3846f009be844b4ab3849d5ad442e6cb [term 1 FOLLOWER]: Refusing update from remote peer d3ffb93de8d841159da80de89f7171e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:59:42.056699 10519 consensus_queue.cc:1035] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8154a02194f741cbb6596035cec6fcf0" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:59:42.057070 10518 consensus_queue.cc:1035] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3846f009be844b4ab3849d5ad442e6cb" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 34669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:59:42.058758 10403 heartbeater.cc:503] Master 127.2.237.61:38995 was elected leader, sending a full tablet report...
I20260620 22:59:42.059657 10491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:42.059091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.2.237.3:42275)
0620 22:59:42.059303 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:42.059431 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260620 22:59:42.067554 10525 mvcc.cc:205] Tried to move back new op lower bound from 7299057180894953472 to 7299057180597514240. Current Snapshot: MvccSnapshot[applied={T|T < 7299057180894953472}]
I20260620 22:59:42.069825 10524 mvcc.cc:205] Tried to move back new op lower bound from 7299057180894953472 to 7299057180597514240. Current Snapshot: MvccSnapshot[applied={T|T < 7299057180894953472}]
I20260620 22:59:42.070168 10490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:42.068550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.2.237.2:44427)
0620 22:59:42.069878 (+ 1328us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:42.069971 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1218}
I20260620 22:59:42.070446 10491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:42.069524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.2.237.3:57329)
0620 22:59:42.070198 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:42.070282 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":556}
I20260620 22:59:42.071362 10492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:42.070521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.2.237.1:40811)
0620 22:59:42.071133 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:42.071216 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":509}
I20260620 22:59:42.074800 10490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:42.074267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.2.237.2:53477)
0620 22:59:42.074529 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:42.074633 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260620 22:59:42.075325 10492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:42.074724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.2.237.1:35903)
0620 22:59:42.075094 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:42.075172 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245}
I20260620 22:59:42.077729 9747 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 22:59:42.078087 9747 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 22:59:42.083688 9747 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 65dcb1d2184d4d41bb04de52e7e5e2aa: Sending DeleteTablet for 3 replicas of tablet fa4f52b3bc5e49358119e078c411af7d
I20260620 22:59:42.084802 10336 tablet_service.cc:1514] Processing DeleteTablet for tablet fa4f52b3bc5e49358119e078c411af7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:42 UTC) from {username='oryx', principal='oryx/127.2.237.61@KRBTEST.COM'} at 127.0.0.1:53134
I20260620 22:59:42.085520 10531 tablet_replica.cc:330] T fa4f52b3bc5e49358119e078c411af7d P 3846f009be844b4ab3849d5ad442e6cb: stopping tablet replica
I20260620 22:59:42.085506 10049 tablet_service.cc:1514] Processing DeleteTablet for tablet fa4f52b3bc5e49358119e078c411af7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:42 UTC) from {username='oryx', principal='oryx/127.2.237.61@KRBTEST.COM'} at 127.0.0.1:47092
I20260620 22:59:42.085728 10531 raft_consensus.cc:2229] T fa4f52b3bc5e49358119e078c411af7d P 3846f009be844b4ab3849d5ad442e6cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 22:59:42.086076 10531 raft_consensus.cc:2258] T fa4f52b3bc5e49358119e078c411af7d P 3846f009be844b4ab3849d5ad442e6cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:42.086140 10532 tablet_replica.cc:330] T fa4f52b3bc5e49358119e078c411af7d P 8154a02194f741cbb6596035cec6fcf0: stopping tablet replica
I20260620 22:59:42.086370 10532 raft_consensus.cc:2229] T fa4f52b3bc5e49358119e078c411af7d P 8154a02194f741cbb6596035cec6fcf0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 22:59:42.086694 10532 raft_consensus.cc:2258] T fa4f52b3bc5e49358119e078c411af7d P 8154a02194f741cbb6596035cec6fcf0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:42.086663 10195 tablet_service.cc:1514] Processing DeleteTablet for tablet fa4f52b3bc5e49358119e078c411af7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:42 UTC) from {username='oryx', principal='oryx/127.2.237.61@KRBTEST.COM'} at 127.0.0.1:33948
I20260620 22:59:42.087284 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 9972
I20260620 22:59:42.087273 10531 ts_tablet_manager.cc:1905] T fa4f52b3bc5e49358119e078c411af7d P 3846f009be844b4ab3849d5ad442e6cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:42.088056 10532 ts_tablet_manager.cc:1905] T fa4f52b3bc5e49358119e078c411af7d P 8154a02194f741cbb6596035cec6fcf0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:42.094497 10531 ts_tablet_manager.cc:1918] T fa4f52b3bc5e49358119e078c411af7d P 3846f009be844b4ab3849d5ad442e6cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 22:59:42.094714 10531 log.cc:1196] T fa4f52b3bc5e49358119e078c411af7d P 3846f009be844b4ab3849d5ad442e6cb: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-2/wal/wals/fa4f52b3bc5e49358119e078c411af7d
I20260620 22:59:42.095320 10531 ts_tablet_manager.cc:1939] T fa4f52b3bc5e49358119e078c411af7d P 3846f009be844b4ab3849d5ad442e6cb: Deleting consensus metadata
I20260620 22:59:42.096807 9732 catalog_manager.cc:4841] TS 3846f009be844b4ab3849d5ad442e6cb (127.2.237.3:34669): tablet fa4f52b3bc5e49358119e078c411af7d (table test-table [id=06015362d9d1410882e7b7324220b73f]) successfully deleted
I20260620 22:59:42.096901 10532 ts_tablet_manager.cc:1918] T fa4f52b3bc5e49358119e078c411af7d P 8154a02194f741cbb6596035cec6fcf0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 22:59:42.097043 10532 log.cc:1196] T fa4f52b3bc5e49358119e078c411af7d P 8154a02194f741cbb6596035cec6fcf0: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-0/wal/wals/fa4f52b3bc5e49358119e078c411af7d
I20260620 22:59:42.097579 10532 ts_tablet_manager.cc:1939] T fa4f52b3bc5e49358119e078c411af7d P 8154a02194f741cbb6596035cec6fcf0: Deleting consensus metadata
I20260620 22:59:42.098819 9732 catalog_manager.cc:4841] TS 8154a02194f741cbb6596035cec6fcf0 (127.2.237.1:41217): tablet fa4f52b3bc5e49358119e078c411af7d (table test-table [id=06015362d9d1410882e7b7324220b73f]) successfully deleted
I20260620 22:59:42.104063 10533 tablet_replica.cc:330] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4: stopping tablet replica
I20260620 22:59:42.104281 10533 raft_consensus.cc:2229] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4 [term 1 LEADER]: Raft consensus shutting down.
I20260620 22:59:42.104635 10533 raft_consensus.cc:2258] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:42.105583 10533 ts_tablet_manager.cc:1905] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:42.110671 10533 ts_tablet_manager.cc:1918] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 22:59:42.110809 10533 log.cc:1196] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781996318807807-2996-0/minicluster-data/ts-1/wal/wals/fa4f52b3bc5e49358119e078c411af7d
I20260620 22:59:42.111209 10533 ts_tablet_manager.cc:1939] T fa4f52b3bc5e49358119e078c411af7d P d3ffb93de8d841159da80de89f7171e4: Deleting consensus metadata
I20260620 22:59:42.112191 9733 catalog_manager.cc:4841] TS d3ffb93de8d841159da80de89f7171e4 (127.2.237.2:33799): tablet fa4f52b3bc5e49358119e078c411af7d (table test-table [id=06015362d9d1410882e7b7324220b73f]) successfully deleted
I20260620 22:59:42.116983 10507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:42.090162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47098 (local address 127.2.237.1:41217)
0620 22:59:42.090348 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:42.090355 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:42.103090 (+ 12735us) server_negotiation.cc:407] Connection header received
0620 22:59:42.103210 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:42.103219 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:42.103327 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:42.103455 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:42.105102 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:42.106061 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:42.107873 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:42.108509 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:42.108654 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:42.111883 (+ 3229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:42.111901 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:42.111925 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:42.111982 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:42.114684 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:42.115266 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:42.115272 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:42.115275 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:42.115387 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:42.115679 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:42.115682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:42.115684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:42.116068 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:42.116270 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:42.116522 (+ 252us) server_negotiation.cc:299] Negotiation successful
0620 22:59:42.116686 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 22:59:42.117676 10109 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:42.210956 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 9972
I20260620 22:59:42.228615 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 10126
I20260620 22:59:42.241657 10509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:42.229349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33966 (local address 127.2.237.2:33799)
0620 22:59:42.229562 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:42.229567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:42.229606 (+ 39us) server_negotiation.cc:407] Connection header received
0620 22:59:42.229773 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:42.229777 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:42.229849 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:42.229950 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:42.231236 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:42.231928 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:42.233135 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:42.233521 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:42.233608 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:42.236125 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:42.236139 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:42.236155 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:42.236188 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:42.239078 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:42.239703 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:42.239709 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:42.239712 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:42.239829 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:42.240140 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:42.240144 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:42.240147 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:42.240599 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:42.240918 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:42.241155 (+ 237us) server_negotiation.cc:299] Negotiation successful
0620 22:59:42.241314 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260620 22:59:42.242316 10255 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:42.304158 10537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:42.303282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.0.0.1:57738)
0620 22:59:42.303870 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:42.303974 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":266,"threads_started":1}
I20260620 22:59:42.305482 10537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:42.305162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:36895 (local address 127.0.0.1:57750)
0620 22:59:42.305300 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:42.305379 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:36895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260620 22:59:42.367180 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 10126
I20260620 22:59:42.389629 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 10273
I20260620 22:59:42.402742 10539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:42.390383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53150 (local address 127.2.237.3:34669)
0620 22:59:42.390894 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:42.390899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:42.390914 (+ 15us) server_negotiation.cc:407] Connection header received
0620 22:59:42.390963 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:42.390967 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:42.391032 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:42.391148 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:42.392458 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:42.393165 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:42.394374 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:42.394808 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:42.394936 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:42.397470 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:42.397485 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:42.397502 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:42.397538 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:42.400272 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:42.400917 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:42.400924 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:42.400926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:42.401073 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:42.401370 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:42.401373 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:42.401375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:42.401704 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:42.401912 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:42.402154 (+ 242us) server_negotiation.cc:299] Negotiation successful
0620 22:59:42.402328 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":291,"threads_started":1}
I20260620 22:59:42.403396 10396 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:42.497630 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 10273
I20260620 22:59:42.514531 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 9610
I20260620 22:59:42.515709 9777 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:42.636962 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 9610
W20260620 22:59:42.658538 9920 connection.cc:535] server connection from 127.0.0.1:42764 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 22:59:42.658916 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 9711
I20260620 22:59:42.660036 9965 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:42.776113 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 9711
2026-06-20T22:59:42Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (11263 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 22:59:42 dist-test-slave-f63b krb5kdc[10545](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 22:59:42 dist-test-slave-f63b krb5kdc[10545](info): set up 2 sockets
Jun 20 22:59:42 dist-test-slave-f63b krb5kdc[10545](info): commencing operation
krb5kdc: starting...
W20260620 22:59:43.867616 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.000s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 22:59:44 dist-test-slave-f63b krb5kdc[10545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T22:59:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T22:59:44Z Disabled control of system clock
WARNING: no policy specified for oryx/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.62.keytab.
Entry for principal oryx/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.62.keytab.
I20260620 22:59:44.081334 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:42559
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.62.keytab
--principal=oryx/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:32999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:42559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:44.408403 10558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:44.408759 10558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:44.408929 10558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:44.416236 10558 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 22:59:44.416383 10558 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260620 22:59:44.416484 10558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:44.416610 10558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:44.416698 10558 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 22:59:44.416786 10558 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 22:59:44.427646 10558 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:32999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:42559
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.62.keytab
--principal=oryx/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:42559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:44.429051 10558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:44.430544 10558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:44.442996 10566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:44.442917 10564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:44.443486 10563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:44.444489 10558 server_base.cc:1034] running on GCE node
I20260620 22:59:44.445287 10558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:44.446738 10558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:44.447996 10558 hybrid_clock.cc:648] HybridClock initialized: now 1781996384447929 us; error 37 us; skew 500 ppm
I20260620 22:59:44.448362 10558 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:44 dist-test-slave-f63b krb5kdc[10545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996384, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:44.451510 10558 init.cc:377] Logged in from keytab as oryx/127.2.237.62@KRBTEST.COM (short username oryx)
I20260620 22:59:44.452976 10558 webserver.cc:460] Webserver started at http://127.2.237.62:33793/ using document root <none> and password file <none>
I20260620 22:59:44.453805 10558 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:44.453949 10558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:44.454290 10558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:44.457412 10558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "cbbbed944e234507bd103f29eeb1325c"
format_stamp: "Formatted at 2026-06-20 22:59:44 on dist-test-slave-f63b"
server_key: "3b5f28ca2b3a9e5c3ffcf55ba2c45d0b"
server_key_iv: "9e2b1600a544df6775e103cef648441f"
server_key_version: "encryptionkey@0"
I20260620 22:59:44.458200 10558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "cbbbed944e234507bd103f29eeb1325c"
format_stamp: "Formatted at 2026-06-20 22:59:44 on dist-test-slave-f63b"
server_key: "3b5f28ca2b3a9e5c3ffcf55ba2c45d0b"
server_key_iv: "9e2b1600a544df6775e103cef648441f"
server_key_version: "encryptionkey@0"
I20260620 22:59:44.463251 10558 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260620 22:59:44.466681 10573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:44.468055 10558 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260620 22:59:44.468266 10558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "cbbbed944e234507bd103f29eeb1325c"
format_stamp: "Formatted at 2026-06-20 22:59:44 on dist-test-slave-f63b"
server_key: "3b5f28ca2b3a9e5c3ffcf55ba2c45d0b"
server_key_iv: "9e2b1600a544df6775e103cef648441f"
server_key_version: "encryptionkey@0"
I20260620 22:59:44.468482 10558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:44.496143 10558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:44.500232 10558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:44.500567 10558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:44.521848 10558 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:42559
I20260620 22:59:44.521872 10634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:42559 every 8 connection(s)
I20260620 22:59:44.523281 10558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 22:59:44.527014 10635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:44.531661 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 10558
I20260620 22:59:44.531864 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 22:59:44.532657 2996 external_mini_cluster.cc:1428] Setting key 117502e00110b47615d6df7188ee7721
I20260620 22:59:44.538165 10635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbbbed944e234507bd103f29eeb1325c: Bootstrap starting.
I20260620 22:59:44.541858 10635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbbbed944e234507bd103f29eeb1325c: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:44.543134 10635 log.cc:824] T 00000000000000000000000000000000 P cbbbed944e234507bd103f29eeb1325c: Log is configured to *not* fsync() on all Append() calls
I20260620 22:59:44.546598 10635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbbbed944e234507bd103f29eeb1325c: No bootstrap required, opened a new log
Jun 20 22:59:44 dist-test-slave-f63b krb5kdc[10545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM
I20260620 22:59:44.554104 10635 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbbbed944e234507bd103f29eeb1325c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbbbed944e234507bd103f29eeb1325c" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 42559 } }
I20260620 22:59:44.554497 10635 raft_consensus.cc:374] T 00000000000000000000000000000000 P cbbbed944e234507bd103f29eeb1325c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:44.554672 10635 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbbbed944e234507bd103f29eeb1325c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbbbed944e234507bd103f29eeb1325c, State: Initialized, Role: FOLLOWER
I20260620 22:59:44.555351 10635 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbbbed944e234507bd103f29eeb1325c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbbbed944e234507bd103f29eeb1325c" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 42559 } }
I20260620 22:59:44.555616 10635 raft_consensus.cc:388] T 00000000000000000000000000000000 P cbbbed944e234507bd103f29eeb1325c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 22:59:44.555763 10635 raft_consensus.cc:482] T 00000000000000000000000000000000 P cbbbed944e234507bd103f29eeb1325c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 22:59:44.555959 10635 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbbbed944e234507bd103f29eeb1325c [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:44.556730 10638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:44.534776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45482 (local address 127.2.237.62:42559)
0620 22:59:44.535612 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:44.535623 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:44.535667 (+ 44us) server_negotiation.cc:407] Connection header received
0620 22:59:44.537332 (+ 1665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:44.537353 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:44.537749 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:44.538057 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:44.539793 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:44.541147 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:44.542968 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:44.543896 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:44.544106 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:44.548669 (+ 4563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:44.548677 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:44.548692 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:44.548729 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:44.552057 (+ 3328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:44.552939 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:44.552946 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:44.552953 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:44.553144 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:44.553553 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:44.553557 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:44.553559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:44.554075 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:44.554395 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:44.554826 (+ 431us) server_negotiation.cc:299] Negotiation successful
0620 22:59:44.555288 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":391,"threads_started":1}
I20260620 22:59:44.558643 10635 raft_consensus.cc:504] T 00000000000000000000000000000000 P cbbbed944e234507bd103f29eeb1325c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbbbed944e234507bd103f29eeb1325c" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 42559 } }
I20260620 22:59:44.559140 10635 leader_election.cc:302] T 00000000000000000000000000000000 P cbbbed944e234507bd103f29eeb1325c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cbbbed944e234507bd103f29eeb1325c; no voters:
I20260620 22:59:44.559795 10635 leader_election.cc:288] T 00000000000000000000000000000000 P cbbbed944e234507bd103f29eeb1325c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 22:59:44.559902 10640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cbbbed944e234507bd103f29eeb1325c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:59:44.560676 10640 raft_consensus.cc:686] T 00000000000000000000000000000000 P cbbbed944e234507bd103f29eeb1325c [term 1 LEADER]: Becoming Leader. State: Replica: cbbbed944e234507bd103f29eeb1325c, State: Running, Role: LEADER
I20260620 22:59:44.561303 10640 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbbbed944e234507bd103f29eeb1325c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbbbed944e234507bd103f29eeb1325c" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 42559 } }
I20260620 22:59:44.562095 10635 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbbbed944e234507bd103f29eeb1325c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 22:59:44.564386 10641 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbbbed944e234507bd103f29eeb1325c [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbbbed944e234507bd103f29eeb1325c. Latest consensus state: current_term: 1 leader_uuid: "cbbbed944e234507bd103f29eeb1325c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbbbed944e234507bd103f29eeb1325c" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 42559 } } }
I20260620 22:59:44.564465 10642 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbbbed944e234507bd103f29eeb1325c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cbbbed944e234507bd103f29eeb1325c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbbbed944e234507bd103f29eeb1325c" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 42559 } } }
I20260620 22:59:44.564803 10641 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbbbed944e234507bd103f29eeb1325c [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:44.564832 10642 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbbbed944e234507bd103f29eeb1325c [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:44.568092 10647 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 22:59:44.573421 10647 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 22:59:44.582762 10647 catalog_manager.cc:1347] Generated new cluster ID: cb65657f21874a7b8f82b5e83c8c9b5c
I20260620 22:59:44.582901 10647 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 22:59:44.593992 10647 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 22:59:44.595611 10647 catalog_manager.cc:1504] Loading token signing keys...
I20260620 22:59:44.602135 10647 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cbbbed944e234507bd103f29eeb1325c: Generated new TSK 0
I20260620 22:59:44.603018 10647 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.2.237.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.237.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.237.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.1.keytab.
Entry for principal oryx/127.2.237.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.1.keytab.
I20260620 22:59:44.659587 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:0
--local_ip_for_outbound_sockets=127.2.237.1
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:42559
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.1.keytab
--principal=oryx/127.2.237.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:32999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:44.993218 10661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:44.993512 10661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:44.993613 10661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:45.000929 10661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:45.001044 10661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:45.001129 10661 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 22:59:45.001211 10661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 22:59:45.011310 10661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:32999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.1.keytab
--principal=oryx/127.2.237.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.237.62:42559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:45.012624 10661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:45.013896 10661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:45.027355 10667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:45.027438 10666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:45.027468 10669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:45.028156 10661 server_base.cc:1034] running on GCE node
I20260620 22:59:45.028867 10661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:45.029824 10661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:45.031071 10661 hybrid_clock.cc:648] HybridClock initialized: now 1781996385031000 us; error 51 us; skew 500 ppm
I20260620 22:59:45.031394 10661 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:45 dist-test-slave-f63b krb5kdc[10545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996385, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:45.039000 10661 init.cc:377] Logged in from keytab as oryx/127.2.237.1@KRBTEST.COM (short username oryx)
I20260620 22:59:45.040393 10661 webserver.cc:460] Webserver started at http://127.2.237.1:40299/ using document root <none> and password file <none>
I20260620 22:59:45.041064 10661 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:45.041148 10661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:45.041406 10661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:45.044286 10661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-0/data/instance:
uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6"
format_stamp: "Formatted at 2026-06-20 22:59:45 on dist-test-slave-f63b"
server_key: "480032ae2e9c5cb36bff24ce09bacd60"
server_key_iv: "22118aa8870c1492dd7c71f28a0529a5"
server_key_version: "encryptionkey@0"
I20260620 22:59:45.045126 10661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance:
uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6"
format_stamp: "Formatted at 2026-06-20 22:59:45 on dist-test-slave-f63b"
server_key: "480032ae2e9c5cb36bff24ce09bacd60"
server_key_iv: "22118aa8870c1492dd7c71f28a0529a5"
server_key_version: "encryptionkey@0"
I20260620 22:59:45.050349 10661 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260620 22:59:45.053980 10676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:45.055537 10661 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 22:59:45.055743 10661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6"
format_stamp: "Formatted at 2026-06-20 22:59:45 on dist-test-slave-f63b"
server_key: "480032ae2e9c5cb36bff24ce09bacd60"
server_key_iv: "22118aa8870c1492dd7c71f28a0529a5"
server_key_version: "encryptionkey@0"
I20260620 22:59:45.056003 10661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:45.070801 10661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:45.075304 10661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:45.075529 10661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:45.077986 10661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:45.078109 10661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:45.078277 10661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:45.078388 10661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 22:59:45 dist-test-slave-f63b krb5kdc[10545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996385, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM
I20260620 22:59:45.102418 10685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:45.078812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:45971 (local address 127.2.237.62:42559)
0620 22:59:45.081160 (+ 2348us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:45.081169 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:45.081193 (+ 24us) server_negotiation.cc:407] Connection header received
0620 22:59:45.082871 (+ 1678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:45.082877 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:45.082968 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:45.083084 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:45.085348 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:45.086712 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:45.089188 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:45.090001 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:45.090151 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:45.095404 (+ 5253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:45.095411 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:45.095416 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:45.095458 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:45.098527 (+ 3069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:45.099605 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:45.099610 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:45.099614 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:45.099740 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:45.100200 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:45.100217 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:45.100221 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:45.100496 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:45.100644 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:45.101487 (+ 843us) server_negotiation.cc:299] Negotiation successful
0620 22:59:45.101826 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2188,"thread_start_us":2088,"threads_started":1}
I20260620 22:59:45.103621 10686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:45.079449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:42559 (local address 127.2.237.1:45971)
0620 22:59:45.080326 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:45.080382 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:45.082545 (+ 2163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:45.083388 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:45.083402 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:45.084254 (+ 852us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:45.085120 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:45.085153 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:45.086890 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:45.086902 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:45.088986 (+ 2084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:45.089022 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:45.090311 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:45.090318 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:45.090503 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:45.091453 (+ 950us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:45.091474 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:45.095161 (+ 3687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:45.098701 (+ 3540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:45.098710 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:45.098732 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:45.099431 (+ 699us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:45.099852 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:45.099857 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:45.099860 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:45.100083 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:45.100626 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:45.100634 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:45.100996 (+ 362us) client_negotiation.cc:769] Sending connection context
0620 22:59:45.101340 (+ 344us) client_negotiation.cc:240] Negotiation successful
0620 22:59:45.102041 (+ 701us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":381,"threads_started":1}
I20260620 22:59:45.128243 10661 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:44539
I20260620 22:59:45.128263 10796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:44539 every 8 connection(s)
I20260620 22:59:45.129554 10661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 22:59:45.131136 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 10661
I20260620 22:59:45.131708 10797 heartbeater.cc:347] Connected to a master server at 127.2.237.62:42559
I20260620 22:59:45.131847 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance
I20260620 22:59:45.132009 10797 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:45.132884 2996 external_mini_cluster.cc:1428] Setting key 622a188404b6769941d50ee42390e74a
I20260620 22:59:45.132887 10797 heartbeater.cc:511] Master 127.2.237.62:42559 requested a full tablet report, sending...
I20260620 22:59:45.135291 10590 ts_manager.cc:194] Registered new tserver with Master: 5cc6e2cea39945e0b6f1c53d5b2b93d6 (127.2.237.1:44539)
I20260620 22:59:45.136358 10789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:45.121718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:42559 (local address 127.2.237.1:51695)
0620 22:59:45.122422 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:45.122440 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:45.122618 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:45.123047 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:45.123050 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:45.123485 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:45.123769 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:45.123778 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:45.125379 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:45.125389 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:45.126809 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:45.126821 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:45.127751 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:45.127756 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:45.127935 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:45.128587 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:45.128604 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:45.129351 (+ 747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:45.133322 (+ 3971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:45.133327 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:45.133331 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:45.133911 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:45.134397 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:45.134402 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:45.134405 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:45.134567 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:45.135235 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:45.135239 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:45.135442 (+ 203us) client_negotiation.cc:769] Sending connection context
0620 22:59:45.135563 (+ 121us) client_negotiation.cc:240] Negotiation successful
0620 22:59:45.135898 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":379,"threads_started":1}
I20260620 22:59:45.137357 10685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:45.121748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:51695 (local address 127.2.237.62:42559)
0620 22:59:45.121982 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:45.121990 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:45.122542 (+ 552us) server_negotiation.cc:407] Connection header received
0620 22:59:45.122768 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:45.122777 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:45.122885 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:45.123032 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:45.123932 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:45.125220 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:45.126950 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:45.127544 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:45.127698 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:45.129561 (+ 1863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:45.129566 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:45.129570 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:45.129623 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:45.133171 (+ 3548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:45.134074 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:45.134079 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:45.134085 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:45.134259 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:45.134749 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:45.134754 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:45.134756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:45.135094 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:45.136714 (+ 1620us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:45.136825 (+ 111us) server_negotiation.cc:299] Negotiation successful
0620 22:59:45.137029 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260620 22:59:45.137310 10590 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.237.1@KRBTEST.COM'} at 127.2.237.1:45971
WARNING: no policy specified for oryx/127.2.237.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.237.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.237.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.2.keytab.
Entry for principal oryx/127.2.237.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.2.keytab.
I20260620 22:59:45.183809 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:0
--local_ip_for_outbound_sockets=127.2.237.2
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:42559
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.2.keytab
--principal=oryx/127.2.237.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:32999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:45.503008 10801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:45.503268 10801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:45.503362 10801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:45.510607 10801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:45.510717 10801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:45.510799 10801 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 22:59:45.510878 10801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 22:59:45.520673 10801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:32999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.2.keytab
--principal=oryx/127.2.237.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.237.62:42559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:45.522254 10801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:45.523977 10801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:45.538388 10807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:45.538945 10806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:45.538878 10809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:45.539426 10801 server_base.cc:1034] running on GCE node
I20260620 22:59:45.540009 10801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:45.540934 10801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:45.542196 10801 hybrid_clock.cc:648] HybridClock initialized: now 1781996385542123 us; error 49 us; skew 500 ppm
I20260620 22:59:45.542596 10801 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:45 dist-test-slave-f63b krb5kdc[10545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996385, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:45.545825 10801 init.cc:377] Logged in from keytab as oryx/127.2.237.2@KRBTEST.COM (short username oryx)
I20260620 22:59:45.547243 10801 webserver.cc:460] Webserver started at http://127.2.237.2:38285/ using document root <none> and password file <none>
I20260620 22:59:45.547869 10801 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:45.547955 10801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:45.548225 10801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:45.551061 10801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-1/data/instance:
uuid: "f35e4a5b9689458a9d8f7dc96f77914a"
format_stamp: "Formatted at 2026-06-20 22:59:45 on dist-test-slave-f63b"
server_key: "6e3d3d5846a79e1c172000e27f2d5b7c"
server_key_iv: "86d9fcb6e3fa66d862639dea12154e4b"
server_key_version: "encryptionkey@0"
I20260620 22:59:45.551779 10801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance:
uuid: "f35e4a5b9689458a9d8f7dc96f77914a"
format_stamp: "Formatted at 2026-06-20 22:59:45 on dist-test-slave-f63b"
server_key: "6e3d3d5846a79e1c172000e27f2d5b7c"
server_key_iv: "86d9fcb6e3fa66d862639dea12154e4b"
server_key_version: "encryptionkey@0"
I20260620 22:59:45.556896 10801 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260620 22:59:45.560513 10816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:45.561820 10801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 22:59:45.561993 10801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "f35e4a5b9689458a9d8f7dc96f77914a"
format_stamp: "Formatted at 2026-06-20 22:59:45 on dist-test-slave-f63b"
server_key: "6e3d3d5846a79e1c172000e27f2d5b7c"
server_key_iv: "86d9fcb6e3fa66d862639dea12154e4b"
server_key_version: "encryptionkey@0"
I20260620 22:59:45.562227 10801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:45.581097 10801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:45.585381 10801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:45.585608 10801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:45.588182 10801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:45.588299 10801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:45.588479 10801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:45.588558 10801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 22:59:45 dist-test-slave-f63b krb5kdc[10545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996385, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM
I20260620 22:59:45.610906 10685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:45.589028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:36347 (local address 127.2.237.62:42559)
0620 22:59:45.589197 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:45.589206 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:45.590631 (+ 1425us) server_negotiation.cc:407] Connection header received
0620 22:59:45.592634 (+ 2003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:45.592640 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:45.592773 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:45.592884 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:45.595538 (+ 2654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:45.596551 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:45.599780 (+ 3229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:45.600333 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:45.600466 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:45.605249 (+ 4783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:45.605254 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:45.605257 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:45.605295 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:45.607858 (+ 2563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:45.608597 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:45.608607 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:45.608609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:45.608718 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:45.609142 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:45.609147 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:45.609150 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:45.609424 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:45.609579 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:45.610308 (+ 729us) server_negotiation.cc:299] Negotiation successful
0620 22:59:45.610537 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260620 22:59:45.612536 10825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:45.589556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:42559 (local address 127.2.237.2:36347)
0620 22:59:45.590446 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:45.590491 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:45.592352 (+ 1861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:45.593249 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:45.593266 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:45.594230 (+ 964us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:45.595308 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:45.595346 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:45.596760 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:45.596769 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:45.599609 (+ 2840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:45.599624 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:45.600596 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:45.600603 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:45.600986 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:45.601832 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:45.601870 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:45.605016 (+ 3146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:45.608004 (+ 2988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:45.608013 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:45.608030 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:45.608468 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:45.608824 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:45.608829 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:45.608833 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:45.609018 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:45.609551 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:45.609559 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:45.609916 (+ 357us) client_negotiation.cc:769] Sending connection context
0620 22:59:45.610188 (+ 272us) client_negotiation.cc:240] Negotiation successful
0620 22:59:45.610742 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":397,"threads_started":1}
I20260620 22:59:45.636034 10685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:45.620357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:45939 (local address 127.2.237.62:42559)
0620 22:59:45.620598 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:45.620606 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:45.621351 (+ 745us) server_negotiation.cc:407] Connection header received
0620 22:59:45.621644 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:45.621648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:45.621729 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:45.621862 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:45.623088 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:45.623847 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:45.625545 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:45.626309 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:45.626476 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:45.629555 (+ 3079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:45.629562 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:45.629567 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:45.629613 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:45.632833 (+ 3220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:45.633464 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:45.633471 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:45.633475 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:45.633635 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:45.634188 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:45.634194 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:45.634199 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:45.634602 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:45.634768 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:45.635360 (+ 592us) server_negotiation.cc:299] Negotiation successful
0620 22:59:45.635616 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260620 22:59:45.636431 10893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:45.620385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:42559 (local address 127.2.237.2:45939)
0620 22:59:45.621219 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:45.621245 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:45.621421 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:45.621892 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:45.621897 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:45.622421 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:45.622919 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:45.622929 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:45.624093 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:45.624102 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:45.625397 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:45.625408 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:45.626526 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:45.626537 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:45.626836 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:45.627671 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:45.627703 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:45.629324 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:45.632963 (+ 3639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:45.632969 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:45.632973 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:45.633347 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:45.633794 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:45.633804 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:45.633807 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:45.634006 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:45.634749 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:45.634754 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:45.635028 (+ 274us) client_negotiation.cc:769] Sending connection context
0620 22:59:45.635249 (+ 221us) client_negotiation.cc:240] Negotiation successful
0620 22:59:45.635705 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":430,"threads_started":1}
I20260620 22:59:45.639250 10801 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:46657
I20260620 22:59:45.639279 10935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:46657 every 8 connection(s)
I20260620 22:59:45.640736 10801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 22:59:45.642729 10936 heartbeater.cc:347] Connected to a master server at 127.2.237.62:42559
I20260620 22:59:45.642949 10936 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:45.643556 10936 heartbeater.cc:511] Master 127.2.237.62:42559 requested a full tablet report, sending...
I20260620 22:59:45.644505 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 10801
I20260620 22:59:45.644693 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance
I20260620 22:59:45.644905 10590 ts_manager.cc:194] Registered new tserver with Master: f35e4a5b9689458a9d8f7dc96f77914a (127.2.237.2:46657)
I20260620 22:59:45.645224 2996 external_mini_cluster.cc:1428] Setting key 441717726c8db4363d0a2ac855077156
I20260620 22:59:45.645881 10590 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.237.2@KRBTEST.COM'} at 127.2.237.2:36347
WARNING: no policy specified for oryx/127.2.237.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.237.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.237.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.3.keytab.
Entry for principal oryx/127.2.237.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.3.keytab.
I20260620 22:59:45.693370 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:0
--local_ip_for_outbound_sockets=127.2.237.3
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:42559
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.3.keytab
--principal=oryx/127.2.237.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:32999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:46.015667 10940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:46.015954 10940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:46.016141 10940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:46.023483 10940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:46.023618 10940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:46.023775 10940 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260620 22:59:46.023902 10940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 22:59:46.033809 10940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:32999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.3.keytab
--principal=oryx/127.2.237.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.237.62:42559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:46.035194 10940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:46.036506 10940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:46.049343 10948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:46.049557 10945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:46.049383 10946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:46.050150 10940 server_base.cc:1034] running on GCE node
I20260620 22:59:46.050791 10940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:46.051637 10940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:46.052845 10940 hybrid_clock.cc:648] HybridClock initialized: now 1781996386052770 us; error 51 us; skew 500 ppm
I20260620 22:59:46.053112 10940 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:46 dist-test-slave-f63b krb5kdc[10545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996386, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:46.056329 10940 init.cc:377] Logged in from keytab as oryx/127.2.237.3@KRBTEST.COM (short username oryx)
I20260620 22:59:46.057711 10940 webserver.cc:460] Webserver started at http://127.2.237.3:37885/ using document root <none> and password file <none>
I20260620 22:59:46.058389 10940 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:46.058516 10940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:46.058871 10940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:46.061817 10940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-2/data/instance:
uuid: "cec82689d41d4bd5a5a588ccfadd4d9f"
format_stamp: "Formatted at 2026-06-20 22:59:46 on dist-test-slave-f63b"
server_key: "55d965824fd027a7d0f0bf1801c73775"
server_key_iv: "8e3d7a5b51fa4bf25166db1367a06518"
server_key_version: "encryptionkey@0"
I20260620 22:59:46.062816 10940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance:
uuid: "cec82689d41d4bd5a5a588ccfadd4d9f"
format_stamp: "Formatted at 2026-06-20 22:59:46 on dist-test-slave-f63b"
server_key: "55d965824fd027a7d0f0bf1801c73775"
server_key_iv: "8e3d7a5b51fa4bf25166db1367a06518"
server_key_version: "encryptionkey@0"
I20260620 22:59:46.068470 10940 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260620 22:59:46.072048 10955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:46.073451 10940 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 22:59:46.073607 10940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "cec82689d41d4bd5a5a588ccfadd4d9f"
format_stamp: "Formatted at 2026-06-20 22:59:46 on dist-test-slave-f63b"
server_key: "55d965824fd027a7d0f0bf1801c73775"
server_key_iv: "8e3d7a5b51fa4bf25166db1367a06518"
server_key_version: "encryptionkey@0"
I20260620 22:59:46.073836 10940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:46.106307 10940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:46.110834 10940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:46.111131 10940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:46.113739 10940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:46.113845 10940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:46.114009 10940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:46.114102 10940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 22:59:46 dist-test-slave-f63b krb5kdc[10545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996386, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM
I20260620 22:59:46.138741 10685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:46.114606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:33679 (local address 127.2.237.62:42559)
0620 22:59:46.114814 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:46.114823 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:46.116251 (+ 1428us) server_negotiation.cc:407] Connection header received
0620 22:59:46.119369 (+ 3118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:46.119375 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:46.119486 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:46.119606 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:46.121754 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.122815 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.125234 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.125844 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.125982 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:46.131222 (+ 5240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:46.131228 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:46.131233 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:46.131271 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:46.135227 (+ 3956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:46.136219 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:46.136226 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:46.136232 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:46.136369 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:46.136883 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:46.136891 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:46.136897 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:46.137177 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:46.137282 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:46.138129 (+ 847us) server_negotiation.cc:299] Negotiation successful
0620 22:59:46.138354 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260620 22:59:46.140038 10964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:46.115158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:42559 (local address 127.2.237.3:33679)
0620 22:59:46.116064 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:46.116115 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:46.118975 (+ 2860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:46.119857 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:46.119865 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:46.120629 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:46.121537 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:46.121572 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.122997 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.123002 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:46.125085 (+ 2083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:46.125097 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.126100 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.126108 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:46.126385 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:46.127285 (+ 900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:46.127323 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:46.130943 (+ 3620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:46.135398 (+ 4455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:46.135412 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:46.135429 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:46.136050 (+ 621us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:46.136501 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:46.136505 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:46.136507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:46.136738 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:46.137377 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:46.137383 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:46.137721 (+ 338us) client_negotiation.cc:769] Sending connection context
0620 22:59:46.138012 (+ 291us) client_negotiation.cc:240] Negotiation successful
0620 22:59:46.138656 (+ 644us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":373,"threads_started":1}
I20260620 22:59:46.142235 10797 heartbeater.cc:503] Master 127.2.237.62:42559 was elected leader, sending a full tablet report...
I20260620 22:59:46.162315 10940 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:35123
I20260620 22:59:46.162356 11074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:35123 every 8 connection(s)
I20260620 22:59:46.163636 10940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 22:59:46.165556 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 10940
I20260620 22:59:46.165752 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance
I20260620 22:59:46.166419 2996 external_mini_cluster.cc:1428] Setting key 7ff34fa865fa0d8dfada95322bed1d5f
I20260620 22:59:46.167143 11075 heartbeater.cc:347] Connected to a master server at 127.2.237.62:42559
I20260620 22:59:46.167460 11075 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:46.168434 11075 heartbeater.cc:511] Master 127.2.237.62:42559 requested a full tablet report, sending...
I20260620 22:59:46.170017 10589 ts_manager.cc:194] Registered new tserver with Master: cec82689d41d4bd5a5a588ccfadd4d9f (127.2.237.3:35123)
I20260620 22:59:46.171204 10589 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.237.3@KRBTEST.COM'} at 127.2.237.3:33679
I20260620 22:59:46.179481 10685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:46.157309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:40143 (local address 127.2.237.62:42559)
0620 22:59:46.157527 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:46.157535 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:46.167127 (+ 9592us) server_negotiation.cc:407] Connection header received
0620 22:59:46.167331 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:46.167337 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:46.167412 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:46.167522 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:46.168874 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.169924 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.171762 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.172222 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.172364 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:46.174378 (+ 2014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:46.174386 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:46.174395 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:46.174423 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:46.176916 (+ 2493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:46.177563 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:46.177570 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:46.177573 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:46.177700 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:46.178044 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:46.178050 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:46.178053 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:46.178317 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:46.178469 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:46.178910 (+ 441us) server_negotiation.cc:299] Negotiation successful
0620 22:59:46.179119 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260620 22:59:46.179556 11063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:46.157357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:42559 (local address 127.2.237.3:40143)
0620 22:59:46.167014 (+ 9657us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:46.167036 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:46.167212 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:46.167716 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:46.167720 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:46.168243 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:46.168670 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:46.168685 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.170110 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.170114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:46.171630 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:46.171641 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.172403 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.172409 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:46.172656 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:46.173326 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:46.173343 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:46.174198 (+ 855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:46.177069 (+ 2871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:46.177074 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:46.177078 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:46.177444 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:46.177808 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:46.177814 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:46.177817 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:46.177939 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:46.178431 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:46.178436 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:46.178662 (+ 226us) client_negotiation.cc:769] Sending connection context
0620 22:59:46.178811 (+ 149us) client_negotiation.cc:240] Negotiation successful
0620 22:59:46.179111 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9504,"mutex_wait_us":1,"thread_start_us":9356,"threads_started":1}
I20260620 22:59:46.181849 2996 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 20 22:59:46 dist-test-slave-f63b krb5kdc[10545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996386, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 20 22:59:46 dist-test-slave-f63b krb5kdc[10545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 22:59:46 dist-test-slave-f63b krb5kdc[10545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
W20260620 22:59:46.230688 10685 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 22:59:46.220112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45486 (local address 127.2.237.62:42559)
0620 22:59:46.220356 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:46.220363 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:46.220571 (+ 208us) server_negotiation.cc:407] Connection header received
0620 22:59:46.220808 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:46.220813 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:46.220875 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:46.220976 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:46.222251 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.223058 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.224307 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.224719 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.224853 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:46.230332 (+ 5479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45486: BlockingRecv error: recv got EOF from 127.0.0.1:45486 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Jun 20 22:59:46 dist-test-slave-f63b krb5kdc[10545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996386, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM
I20260620 22:59:46.247293 10685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:46.233386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45494 (local address 127.2.237.62:42559)
0620 22:59:46.233623 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:46.233628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:46.233856 (+ 228us) server_negotiation.cc:407] Connection header received
0620 22:59:46.233963 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:46.233967 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:46.234043 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:46.234128 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:46.235341 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.236093 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.237294 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.237719 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.237819 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:46.241979 (+ 4160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:46.241984 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:46.241987 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:46.242037 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:46.244733 (+ 2696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:46.245400 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:46.245404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:46.245407 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:46.245524 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:46.245890 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:46.245894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:46.245897 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:46.246163 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:46.246457 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:46.246766 (+ 309us) server_negotiation.cc:299] Negotiation successful
0620 22:59:46.246982 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260620 22:59:46.251431 10590 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45494:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 22:59:46.258843 10590 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 22:59:46.283588 11093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:46.271043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:44539 (local address 127.0.0.1:57960)
0620 22:59:46.271959 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:46.272070 (+ 111us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:46.272302 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:46.273521 (+ 1219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:46.273532 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:46.273583 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:59:46.274060 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:46.274073 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.276796 (+ 2723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.276805 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:46.279934 (+ 3129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:46.279960 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.282273 (+ 2313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.282282 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:46.282599 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:46.282682 (+ 83us) client_negotiation.cc:769] Sending connection context
0620 22:59:46.282814 (+ 132us) client_negotiation.cc:240] Negotiation successful
0620 22:59:46.283064 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":423,"threads_started":1}
I20260620 22:59:46.285338 11094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:46.271218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57960 (local address 127.2.237.1:44539)
0620 22:59:46.272877 (+ 1659us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:46.272883 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:46.272905 (+ 22us) server_negotiation.cc:407] Connection header received
0620 22:59:46.272977 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:46.272983 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:46.273258 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:46.274293 (+ 1035us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:59:46.274491 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.276576 (+ 2085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.280210 (+ 3634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.282023 (+ 1813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.284324 (+ 2301us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:46.284617 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:46.284699 (+ 82us) server_negotiation.cc:299] Negotiation successful
0620 22:59:46.284923 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1511,"thread_start_us":1404,"threads_started":1}
I20260620 22:59:46.294450 10727 tablet_service.cc:1467] Processing CreateTablet for tablet dc4c43c988704c9097fcf7e31b310a9f (DEFAULT_TABLE table=test-table [id=2ff6639d0b514219a832750cbf6c6872]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:59:46.295650 11093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:46.271721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:35123 (local address 127.0.0.1:38116)
0620 22:59:46.283968 (+ 12247us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:46.283994 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:46.284117 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:46.286941 (+ 2824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:46.286947 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:46.286973 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:59:46.287208 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:46.287225 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.291090 (+ 3865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.291098 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:46.292994 (+ 1896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:46.293035 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.294833 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.294838 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:46.295047 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:46.295081 (+ 34us) client_negotiation.cc:769] Sending connection context
0620 22:59:46.295185 (+ 104us) client_negotiation.cc:240] Negotiation successful
0620 22:59:46.295328 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12123,"thread_start_us":18633,"threads_started":1}
I20260620 22:59:46.295750 11097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:46.272379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38116 (local address 127.2.237.3:35123)
0620 22:59:46.286244 (+ 13865us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:46.286253 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:46.286287 (+ 34us) server_negotiation.cc:407] Connection header received
0620 22:59:46.286391 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:46.286401 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:46.286723 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:46.287427 (+ 704us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:59:46.287837 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.290027 (+ 2190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.293202 (+ 3175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.294661 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.294811 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:46.295074 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:46.295247 (+ 173us) server_negotiation.cc:299] Negotiation successful
0620 22:59:46.295417 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13679,"thread_start_us":13458,"threads_started":1}
I20260620 22:59:46.296140 10727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc4c43c988704c9097fcf7e31b310a9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:46.299333 11095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:46.272258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:46657 (local address 127.0.0.1:60774)
0620 22:59:46.290717 (+ 18459us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:46.290741 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:46.290885 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:46.292437 (+ 1552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:46.292441 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:46.292465 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:59:46.293062 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:46.293072 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.294920 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.294925 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:46.296718 (+ 1793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:46.296729 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.298517 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.298525 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:46.298728 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:46.298765 (+ 37us) client_negotiation.cc:769] Sending connection context
0620 22:59:46.298865 (+ 100us) client_negotiation.cc:240] Negotiation successful
0620 22:59:46.299000 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18278,"thread_start_us":18569,"threads_started":1}
I20260620 22:59:46.299721 11005 tablet_service.cc:1467] Processing CreateTablet for tablet dc4c43c988704c9097fcf7e31b310a9f (DEFAULT_TABLE table=test-table [id=2ff6639d0b514219a832750cbf6c6872]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:59:46.300200 11098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:46.273205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60774 (local address 127.2.237.2:46657)
0620 22:59:46.291435 (+ 18230us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:46.291441 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:46.291465 (+ 24us) server_negotiation.cc:407] Connection header received
0620 22:59:46.291543 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:46.291548 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:46.291758 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:46.291979 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:59:46.293284 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.294755 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.296880 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.298268 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.298432 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:46.298637 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:46.299770 (+ 1133us) server_negotiation.cc:299] Negotiation successful
0620 22:59:46.299928 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18075,"thread_start_us":17963,"threads_started":1}
I20260620 22:59:46.300915 11005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc4c43c988704c9097fcf7e31b310a9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:46.303606 10866 tablet_service.cc:1467] Processing CreateTablet for tablet dc4c43c988704c9097fcf7e31b310a9f (DEFAULT_TABLE table=test-table [id=2ff6639d0b514219a832750cbf6c6872]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:59:46.304932 10866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc4c43c988704c9097fcf7e31b310a9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:46.308758 11099 tablet_bootstrap.cc:492] T dc4c43c988704c9097fcf7e31b310a9f P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Bootstrap starting.
I20260620 22:59:46.311750 11099 tablet_bootstrap.cc:654] T dc4c43c988704c9097fcf7e31b310a9f P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:46.312724 11099 log.cc:824] T dc4c43c988704c9097fcf7e31b310a9f P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Log is configured to *not* fsync() on all Append() calls
I20260620 22:59:46.315222 11101 tablet_bootstrap.cc:492] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f: Bootstrap starting.
I20260620 22:59:46.315996 11099 tablet_bootstrap.cc:492] T dc4c43c988704c9097fcf7e31b310a9f P 5cc6e2cea39945e0b6f1c53d5b2b93d6: No bootstrap required, opened a new log
I20260620 22:59:46.316255 11099 ts_tablet_manager.cc:1397] T dc4c43c988704c9097fcf7e31b310a9f P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.002s
I20260620 22:59:46.318941 11101 tablet_bootstrap.cc:654] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:46.320317 11101 log.cc:824] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f: Log is configured to *not* fsync() on all Append() calls
I20260620 22:59:46.321699 11102 tablet_bootstrap.cc:492] T dc4c43c988704c9097fcf7e31b310a9f P f35e4a5b9689458a9d8f7dc96f77914a: Bootstrap starting.
I20260620 22:59:46.323666 11101 tablet_bootstrap.cc:492] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f: No bootstrap required, opened a new log
I20260620 22:59:46.323939 11101 ts_tablet_manager.cc:1397] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260620 22:59:46.325366 11102 tablet_bootstrap.cc:654] T dc4c43c988704c9097fcf7e31b310a9f P f35e4a5b9689458a9d8f7dc96f77914a: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:46.326341 11102 log.cc:824] T dc4c43c988704c9097fcf7e31b310a9f P f35e4a5b9689458a9d8f7dc96f77914a: Log is configured to *not* fsync() on all Append() calls
I20260620 22:59:46.327039 11099 raft_consensus.cc:348] T dc4c43c988704c9097fcf7e31b310a9f P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:46.327484 11099 raft_consensus.cc:374] T dc4c43c988704c9097fcf7e31b310a9f P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:46.327631 11099 raft_consensus.cc:729] T dc4c43c988704c9097fcf7e31b310a9f P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc6e2cea39945e0b6f1c53d5b2b93d6, State: Initialized, Role: FOLLOWER
I20260620 22:59:46.328284 11099 consensus_queue.cc:260] T dc4c43c988704c9097fcf7e31b310a9f P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:46.329974 11099 ts_tablet_manager.cc:1428] T dc4c43c988704c9097fcf7e31b310a9f P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Time spent starting tablet: real 0.014s user 0.011s sys 0.002s
I20260620 22:59:46.330448 11102 tablet_bootstrap.cc:492] T dc4c43c988704c9097fcf7e31b310a9f P f35e4a5b9689458a9d8f7dc96f77914a: No bootstrap required, opened a new log
I20260620 22:59:46.330753 11102 ts_tablet_manager.cc:1397] T dc4c43c988704c9097fcf7e31b310a9f P f35e4a5b9689458a9d8f7dc96f77914a: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260620 22:59:46.333076 11101 raft_consensus.cc:348] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:46.333575 11101 raft_consensus.cc:374] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:46.333686 11101 raft_consensus.cc:729] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cec82689d41d4bd5a5a588ccfadd4d9f, State: Initialized, Role: FOLLOWER
I20260620 22:59:46.334265 11101 consensus_queue.cc:260] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:46.335318 11075 heartbeater.cc:503] Master 127.2.237.62:42559 was elected leader, sending a full tablet report...
I20260620 22:59:46.336937 11101 ts_tablet_manager.cc:1428] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
I20260620 22:59:46.339707 11102 raft_consensus.cc:348] T dc4c43c988704c9097fcf7e31b310a9f P f35e4a5b9689458a9d8f7dc96f77914a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:46.340019 11102 raft_consensus.cc:374] T dc4c43c988704c9097fcf7e31b310a9f P f35e4a5b9689458a9d8f7dc96f77914a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:46.340116 11102 raft_consensus.cc:729] T dc4c43c988704c9097fcf7e31b310a9f P f35e4a5b9689458a9d8f7dc96f77914a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f35e4a5b9689458a9d8f7dc96f77914a, State: Initialized, Role: FOLLOWER
I20260620 22:59:46.340663 11102 consensus_queue.cc:260] T dc4c43c988704c9097fcf7e31b310a9f P f35e4a5b9689458a9d8f7dc96f77914a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:46.341586 10936 heartbeater.cc:503] Master 127.2.237.62:42559 was elected leader, sending a full tablet report...
I20260620 22:59:46.341967 11102 ts_tablet_manager.cc:1428] T dc4c43c988704c9097fcf7e31b310a9f P f35e4a5b9689458a9d8f7dc96f77914a: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
W20260620 22:59:46.392829 10937 tablet.cc:2369] T dc4c43c988704c9097fcf7e31b310a9f P f35e4a5b9689458a9d8f7dc96f77914a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 22:59:46.394232 10798 tablet.cc:2369] T dc4c43c988704c9097fcf7e31b310a9f P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 22:59:46.415482 11076 tablet.cc:2369] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 22:59:46.589107 11106 raft_consensus.cc:482] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 22:59:46.589422 11106 raft_consensus.cc:504] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:46.591128 11106 leader_election.cc:288] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc6e2cea39945e0b6f1c53d5b2b93d6 (127.2.237.1:44539), f35e4a5b9689458a9d8f7dc96f77914a (127.2.237.2:46657)
I20260620 22:59:46.596519 10964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:46.591415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:44539 (local address 127.2.237.3:36871)
0620 22:59:46.591609 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:46.591630 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:46.591752 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:46.592303 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:46.592307 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:46.592328 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:59:46.592504 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:46.592511 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.593455 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.593464 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:46.594730 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:46.594743 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.595795 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.595801 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:46.595997 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:46.596021 (+ 24us) client_negotiation.cc:769] Sending connection context
0620 22:59:46.596128 (+ 107us) client_negotiation.cc:240] Negotiation successful
0620 22:59:46.596252 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260620 22:59:46.596511 11094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:46.591566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:36871 (local address 127.2.237.1:44539)
0620 22:59:46.591794 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:46.591800 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:46.591822 (+ 22us) server_negotiation.cc:407] Connection header received
0620 22:59:46.591908 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:46.591917 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:46.592028 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:46.592132 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:59:46.592648 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.593332 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.594896 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.595645 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.595791 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:46.595892 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:46.596160 (+ 268us) server_negotiation.cc:299] Negotiation successful
0620 22:59:46.596279 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260620 22:59:46.597278 11098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:46.592063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:34423 (local address 127.2.237.2:46657)
0620 22:59:46.592257 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:46.592262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:46.592406 (+ 144us) server_negotiation.cc:407] Connection header received
0620 22:59:46.592620 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:46.592624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:46.592677 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:46.592772 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:59:46.593298 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.594003 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.595637 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.596372 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.596523 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:46.596631 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:46.596927 (+ 296us) server_negotiation.cc:299] Negotiation successful
0620 22:59:46.597052 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260620 22:59:46.597280 11108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:46.591726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:46657 (local address 127.2.237.3:34423)
0620 22:59:46.592314 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:46.592328 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:46.592484 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:46.592784 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:46.592788 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:46.592810 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:59:46.593142 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:46.593149 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.594133 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.594139 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:46.595488 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:46.595499 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.596594 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.596599 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:46.596766 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:46.596793 (+ 27us) client_negotiation.cc:769] Sending connection context
0620 22:59:46.596878 (+ 85us) client_negotiation.cc:240] Negotiation successful
0620 22:59:46.596988 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":325,"threads_started":1}
I20260620 22:59:46.597681 10747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc4c43c988704c9097fcf7e31b310a9f" candidate_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" is_pre_election: true
I20260620 22:59:46.598222 10747 raft_consensus.cc:2454] T dc4c43c988704c9097fcf7e31b310a9f P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cec82689d41d4bd5a5a588ccfadd4d9f in term 0.
I20260620 22:59:46.598261 10890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc4c43c988704c9097fcf7e31b310a9f" candidate_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" is_pre_election: true
I20260620 22:59:46.598793 10890 raft_consensus.cc:2454] T dc4c43c988704c9097fcf7e31b310a9f P f35e4a5b9689458a9d8f7dc96f77914a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cec82689d41d4bd5a5a588ccfadd4d9f in term 0.
I20260620 22:59:46.598920 10959 leader_election.cc:302] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc6e2cea39945e0b6f1c53d5b2b93d6, cec82689d41d4bd5a5a588ccfadd4d9f; no voters:
I20260620 22:59:46.599308 11106 raft_consensus.cc:2781] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 22:59:46.599458 11106 raft_consensus.cc:482] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 22:59:46.599577 11106 raft_consensus.cc:3037] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:46.601861 11106 raft_consensus.cc:504] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:46.602456 11106 leader_election.cc:288] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f [CANDIDATE]: Term 1 election: Requested vote from peers 5cc6e2cea39945e0b6f1c53d5b2b93d6 (127.2.237.1:44539), f35e4a5b9689458a9d8f7dc96f77914a (127.2.237.2:46657)
I20260620 22:59:46.602941 10747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc4c43c988704c9097fcf7e31b310a9f" candidate_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6"
I20260620 22:59:46.603044 10890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc4c43c988704c9097fcf7e31b310a9f" candidate_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35e4a5b9689458a9d8f7dc96f77914a"
I20260620 22:59:46.603142 10747 raft_consensus.cc:3037] T dc4c43c988704c9097fcf7e31b310a9f P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:46.603312 10890 raft_consensus.cc:3037] T dc4c43c988704c9097fcf7e31b310a9f P f35e4a5b9689458a9d8f7dc96f77914a [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:46.605487 10890 raft_consensus.cc:2454] T dc4c43c988704c9097fcf7e31b310a9f P f35e4a5b9689458a9d8f7dc96f77914a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cec82689d41d4bd5a5a588ccfadd4d9f in term 1.
I20260620 22:59:46.605499 10747 raft_consensus.cc:2454] T dc4c43c988704c9097fcf7e31b310a9f P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cec82689d41d4bd5a5a588ccfadd4d9f in term 1.
I20260620 22:59:46.605866 10959 leader_election.cc:302] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cec82689d41d4bd5a5a588ccfadd4d9f, f35e4a5b9689458a9d8f7dc96f77914a; no voters:
I20260620 22:59:46.606173 11106 raft_consensus.cc:2781] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:59:46.606734 11106 raft_consensus.cc:686] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f [term 1 LEADER]: Becoming Leader. State: Replica: cec82689d41d4bd5a5a588ccfadd4d9f, State: Running, Role: LEADER
I20260620 22:59:46.607244 11106 consensus_queue.cc:237] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:46.612078 10590 catalog_manager.cc:5495] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f reported cstate change: term changed from 0 to 1, leader changed from <none> to cec82689d41d4bd5a5a588ccfadd4d9f (127.2.237.3). New cstate: current_term: 1 leader_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } health_report { overall_health: HEALTHY } } }
I20260620 22:59:46.648671 11097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:46.642623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46886 (local address 127.2.237.3:35123)
0620 22:59:46.642863 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:46.642868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:46.642934 (+ 66us) server_negotiation.cc:407] Connection header received
0620 22:59:46.643235 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:46.643238 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:46.643326 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:46.643444 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 22:59:46.643841 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.644635 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.646508 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.646970 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.647062 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:46.647537 (+ 475us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 22:59:46.648097 (+ 560us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 22:59:46.648212 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:46.648339 (+ 127us) server_negotiation.cc:299] Negotiation successful
0620 22:59:46.648403 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260620 22:59:46.657315 10890 raft_consensus.cc:1261] T dc4c43c988704c9097fcf7e31b310a9f P f35e4a5b9689458a9d8f7dc96f77914a [term 1 FOLLOWER]: Refusing update from remote peer cec82689d41d4bd5a5a588ccfadd4d9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:59:46.657430 10747 raft_consensus.cc:1261] T dc4c43c988704c9097fcf7e31b310a9f P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 1 FOLLOWER]: Refusing update from remote peer cec82689d41d4bd5a5a588ccfadd4d9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:59:46.658234 11109 consensus_queue.cc:1035] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:59:46.658679 11106 consensus_queue.cc:1035] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:59:46.672688 11112 mvcc.cc:205] Tried to move back new op lower bound from 7299057199739392000 to 7299057199547408384. Current Snapshot: MvccSnapshot[applied={T|T < 7299057199739392000}]
I20260620 22:59:46.674306 11113 mvcc.cc:205] Tried to move back new op lower bound from 7299057199739392000 to 7299057199547408384. Current Snapshot: MvccSnapshot[applied={T|T < 7299057199739392000}]
I20260620 22:59:46.685981 10590 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 22:59:46.686275 10590 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 22:59:46.689976 10590 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cbbbed944e234507bd103f29eeb1325c: Sending DeleteTablet for 3 replicas of tablet dc4c43c988704c9097fcf7e31b310a9f
I20260620 22:59:46.691063 10727 tablet_service.cc:1514] Processing DeleteTablet for tablet dc4c43c988704c9097fcf7e31b310a9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:46 UTC) from {username='oryx', principal='oryx/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:57960
I20260620 22:59:46.691128 10866 tablet_service.cc:1514] Processing DeleteTablet for tablet dc4c43c988704c9097fcf7e31b310a9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:46 UTC) from {username='oryx', principal='oryx/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:60774
I20260620 22:59:46.691978 11005 tablet_service.cc:1514] Processing DeleteTablet for tablet dc4c43c988704c9097fcf7e31b310a9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:46 UTC) from {username='oryx', principal='oryx/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:38116
I20260620 22:59:46.692483 10590 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45494:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 22:59:46.693493 10590 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 22:59:46.700799 11121 tablet_replica.cc:330] T dc4c43c988704c9097fcf7e31b310a9f P 5cc6e2cea39945e0b6f1c53d5b2b93d6: stopping tablet replica
I20260620 22:59:46.701027 11123 tablet_replica.cc:330] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f: stopping tablet replica
I20260620 22:59:46.701269 11121 raft_consensus.cc:2229] T dc4c43c988704c9097fcf7e31b310a9f P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 22:59:46.701659 11123 raft_consensus.cc:2229] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f [term 1 LEADER]: Raft consensus shutting down.
I20260620 22:59:46.701790 11121 raft_consensus.cc:2258] T dc4c43c988704c9097fcf7e31b310a9f P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:46.703723 11123 raft_consensus.cc:2258] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:46.703887 11122 tablet_replica.cc:330] T dc4c43c988704c9097fcf7e31b310a9f P f35e4a5b9689458a9d8f7dc96f77914a: stopping tablet replica
I20260620 22:59:46.704394 11122 raft_consensus.cc:2229] T dc4c43c988704c9097fcf7e31b310a9f P f35e4a5b9689458a9d8f7dc96f77914a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 22:59:46.704931 11122 raft_consensus.cc:2258] T dc4c43c988704c9097fcf7e31b310a9f P f35e4a5b9689458a9d8f7dc96f77914a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:46.707048 11122 ts_tablet_manager.cc:1905] T dc4c43c988704c9097fcf7e31b310a9f P f35e4a5b9689458a9d8f7dc96f77914a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:46.707067 11123 ts_tablet_manager.cc:1905] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:46.711898 11121 ts_tablet_manager.cc:1905] T dc4c43c988704c9097fcf7e31b310a9f P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:46.713195 10866 tablet_service.cc:1467] Processing CreateTablet for tablet 4e0a71c55ea2406da78c7ad7742d96b7 (DEFAULT_TABLE table=test-table [id=4d2518219de84fdc86bd8e856462835d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:59:46.713649 10866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e0a71c55ea2406da78c7ad7742d96b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:46.714147 11122 ts_tablet_manager.cc:1918] T dc4c43c988704c9097fcf7e31b310a9f P f35e4a5b9689458a9d8f7dc96f77914a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 22:59:46.714351 11122 log.cc:1196] T dc4c43c988704c9097fcf7e31b310a9f P f35e4a5b9689458a9d8f7dc96f77914a: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-1/wal/wals/dc4c43c988704c9097fcf7e31b310a9f
I20260620 22:59:46.715049 11122 ts_tablet_manager.cc:1939] T dc4c43c988704c9097fcf7e31b310a9f P f35e4a5b9689458a9d8f7dc96f77914a: Deleting consensus metadata
I20260620 22:59:46.717007 10577 catalog_manager.cc:4841] TS f35e4a5b9689458a9d8f7dc96f77914a (127.2.237.2:46657): tablet dc4c43c988704c9097fcf7e31b310a9f (table test-table [id=2ff6639d0b514219a832750cbf6c6872]) successfully deleted
I20260620 22:59:46.718101 11123 ts_tablet_manager.cc:1918] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 22:59:46.718312 11123 log.cc:1196] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-2/wal/wals/dc4c43c988704c9097fcf7e31b310a9f
I20260620 22:59:46.718863 11123 ts_tablet_manager.cc:1939] T dc4c43c988704c9097fcf7e31b310a9f P cec82689d41d4bd5a5a588ccfadd4d9f: Deleting consensus metadata
I20260620 22:59:46.719197 11121 ts_tablet_manager.cc:1918] T dc4c43c988704c9097fcf7e31b310a9f P 5cc6e2cea39945e0b6f1c53d5b2b93d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 22:59:46.719352 11121 log.cc:1196] T dc4c43c988704c9097fcf7e31b310a9f P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-0/wal/wals/dc4c43c988704c9097fcf7e31b310a9f
I20260620 22:59:46.719794 11102 tablet_bootstrap.cc:492] T 4e0a71c55ea2406da78c7ad7742d96b7 P f35e4a5b9689458a9d8f7dc96f77914a: Bootstrap starting.
I20260620 22:59:46.719854 11121 ts_tablet_manager.cc:1939] T dc4c43c988704c9097fcf7e31b310a9f P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Deleting consensus metadata
I20260620 22:59:46.720160 10727 tablet_service.cc:1467] Processing CreateTablet for tablet 4e0a71c55ea2406da78c7ad7742d96b7 (DEFAULT_TABLE table=test-table [id=4d2518219de84fdc86bd8e856462835d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:59:46.720568 10727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e0a71c55ea2406da78c7ad7742d96b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:46.721400 11102 tablet_bootstrap.cc:654] T 4e0a71c55ea2406da78c7ad7742d96b7 P f35e4a5b9689458a9d8f7dc96f77914a: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:46.721562 10577 catalog_manager.cc:4841] TS 5cc6e2cea39945e0b6f1c53d5b2b93d6 (127.2.237.1:44539): tablet dc4c43c988704c9097fcf7e31b310a9f (table test-table [id=2ff6639d0b514219a832750cbf6c6872]) successfully deleted
I20260620 22:59:46.722085 10575 catalog_manager.cc:4841] TS cec82689d41d4bd5a5a588ccfadd4d9f (127.2.237.3:35123): tablet dc4c43c988704c9097fcf7e31b310a9f (table test-table [id=2ff6639d0b514219a832750cbf6c6872]) successfully deleted
I20260620 22:59:46.722565 11005 tablet_service.cc:1467] Processing CreateTablet for tablet 4e0a71c55ea2406da78c7ad7742d96b7 (DEFAULT_TABLE table=test-table [id=4d2518219de84fdc86bd8e856462835d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:59:46.723016 11005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e0a71c55ea2406da78c7ad7742d96b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:46.726481 11099 tablet_bootstrap.cc:492] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Bootstrap starting.
I20260620 22:59:46.728052 11099 tablet_bootstrap.cc:654] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:46.730326 11101 tablet_bootstrap.cc:492] T 4e0a71c55ea2406da78c7ad7742d96b7 P cec82689d41d4bd5a5a588ccfadd4d9f: Bootstrap starting.
I20260620 22:59:46.731920 11099 tablet_bootstrap.cc:492] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: No bootstrap required, opened a new log
I20260620 22:59:46.731997 11101 tablet_bootstrap.cc:654] T 4e0a71c55ea2406da78c7ad7742d96b7 P cec82689d41d4bd5a5a588ccfadd4d9f: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:46.732113 11099 ts_tablet_manager.cc:1397] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260620 22:59:46.733131 11102 tablet_bootstrap.cc:492] T 4e0a71c55ea2406da78c7ad7742d96b7 P f35e4a5b9689458a9d8f7dc96f77914a: No bootstrap required, opened a new log
I20260620 22:59:46.732967 11099 raft_consensus.cc:348] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:46.733250 11099 raft_consensus.cc:374] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:46.733374 11102 ts_tablet_manager.cc:1397] T 4e0a71c55ea2406da78c7ad7742d96b7 P f35e4a5b9689458a9d8f7dc96f77914a: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.012s
I20260620 22:59:46.733371 11099 raft_consensus.cc:729] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc6e2cea39945e0b6f1c53d5b2b93d6, State: Initialized, Role: FOLLOWER
I20260620 22:59:46.733717 11099 consensus_queue.cc:260] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:46.734329 11099 ts_tablet_manager.cc:1428] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260620 22:59:46.734288 11102 raft_consensus.cc:348] T 4e0a71c55ea2406da78c7ad7742d96b7 P f35e4a5b9689458a9d8f7dc96f77914a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:46.734494 11102 raft_consensus.cc:374] T 4e0a71c55ea2406da78c7ad7742d96b7 P f35e4a5b9689458a9d8f7dc96f77914a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:46.734688 11102 raft_consensus.cc:729] T 4e0a71c55ea2406da78c7ad7742d96b7 P f35e4a5b9689458a9d8f7dc96f77914a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f35e4a5b9689458a9d8f7dc96f77914a, State: Initialized, Role: FOLLOWER
I20260620 22:59:46.734998 11102 consensus_queue.cc:260] T 4e0a71c55ea2406da78c7ad7742d96b7 P f35e4a5b9689458a9d8f7dc96f77914a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:46.735598 11102 ts_tablet_manager.cc:1428] T 4e0a71c55ea2406da78c7ad7742d96b7 P f35e4a5b9689458a9d8f7dc96f77914a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260620 22:59:46.737082 11101 tablet_bootstrap.cc:492] T 4e0a71c55ea2406da78c7ad7742d96b7 P cec82689d41d4bd5a5a588ccfadd4d9f: No bootstrap required, opened a new log
I20260620 22:59:46.737279 11101 ts_tablet_manager.cc:1397] T 4e0a71c55ea2406da78c7ad7742d96b7 P cec82689d41d4bd5a5a588ccfadd4d9f: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260620 22:59:46.738068 11101 raft_consensus.cc:348] T 4e0a71c55ea2406da78c7ad7742d96b7 P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:46.738315 11101 raft_consensus.cc:374] T 4e0a71c55ea2406da78c7ad7742d96b7 P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:46.738438 11101 raft_consensus.cc:729] T 4e0a71c55ea2406da78c7ad7742d96b7 P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cec82689d41d4bd5a5a588ccfadd4d9f, State: Initialized, Role: FOLLOWER
I20260620 22:59:46.738814 11101 consensus_queue.cc:260] T 4e0a71c55ea2406da78c7ad7742d96b7 P cec82689d41d4bd5a5a588ccfadd4d9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:46.739969 11101 ts_tablet_manager.cc:1428] T 4e0a71c55ea2406da78c7ad7742d96b7 P cec82689d41d4bd5a5a588ccfadd4d9f: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260620 22:59:46.870609 11105 raft_consensus.cc:482] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 22:59:46.870875 11105 raft_consensus.cc:504] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:46.872651 11105 leader_election.cc:288] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f35e4a5b9689458a9d8f7dc96f77914a (127.2.237.2:46657), cec82689d41d4bd5a5a588ccfadd4d9f (127.2.237.3:35123)
I20260620 22:59:46.880817 11097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:46.873701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:44693 (local address 127.2.237.3:35123)
0620 22:59:46.873878 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:46.873883 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:46.874328 (+ 445us) server_negotiation.cc:407] Connection header received
0620 22:59:46.874550 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:46.874575 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:46.874682 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:46.874777 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:59:46.875438 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.876577 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.878475 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.879753 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.879890 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:46.879990 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:46.880350 (+ 360us) server_negotiation.cc:299] Negotiation successful
0620 22:59:46.880480 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260620 22:59:46.880878 11128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:46.873143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:35123 (local address 127.2.237.1:44693)
0620 22:59:46.874235 (+ 1092us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:46.874253 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:46.874416 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:46.874826 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:46.874831 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:46.874855 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:59:46.875276 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:46.875286 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.876711 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.876717 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:46.878303 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:46.878316 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.879958 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.879965 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:46.880167 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:46.880202 (+ 35us) client_negotiation.cc:769] Sending connection context
0620 22:59:46.880290 (+ 88us) client_negotiation.cc:240] Negotiation successful
0620 22:59:46.880430 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":974,"mutex_wait_us":75,"thread_start_us":781,"threads_started":1}
I20260620 22:59:46.881820 11098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:46.873144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:36429 (local address 127.2.237.2:46657)
0620 22:59:46.873357 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:46.873366 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:46.874050 (+ 684us) server_negotiation.cc:407] Connection header received
0620 22:59:46.874397 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:46.874403 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:46.874498 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:46.874634 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 22:59:46.875255 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.876425 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.878995 (+ 2570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.880280 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.880397 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:46.880481 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:46.881410 (+ 929us) server_negotiation.cc:299] Negotiation successful
0620 22:59:46.881547 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260620 22:59:46.881767 11025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e0a71c55ea2406da78c7ad7742d96b7" candidate_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" is_pre_election: true
I20260620 22:59:46.881843 11127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:46.873064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:46657 (local address 127.2.237.1:36429)
0620 22:59:46.873919 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:46.873943 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:46.874183 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:46.874644 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:46.874649 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:46.874672 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 22:59:46.875068 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:46.875079 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.876569 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.876574 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:46.878852 (+ 2278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:46.878868 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.880638 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.880644 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:46.880825 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:46.880871 (+ 46us) client_negotiation.cc:769] Sending connection context
0620 22:59:46.881373 (+ 502us) client_negotiation.cc:240] Negotiation successful
0620 22:59:46.881509 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":558,"threads_started":1}
I20260620 22:59:46.882285 11025 raft_consensus.cc:2454] T 4e0a71c55ea2406da78c7ad7742d96b7 P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc6e2cea39945e0b6f1c53d5b2b93d6 in term 0.
I20260620 22:59:46.882844 10890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e0a71c55ea2406da78c7ad7742d96b7" candidate_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" is_pre_election: true
I20260620 22:59:46.882990 10678 leader_election.cc:302] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc6e2cea39945e0b6f1c53d5b2b93d6, cec82689d41d4bd5a5a588ccfadd4d9f; no voters:
I20260620 22:59:46.883105 10890 raft_consensus.cc:2454] T 4e0a71c55ea2406da78c7ad7742d96b7 P f35e4a5b9689458a9d8f7dc96f77914a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc6e2cea39945e0b6f1c53d5b2b93d6 in term 0.
I20260620 22:59:46.883390 11105 raft_consensus.cc:2781] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 22:59:46.883533 11105 raft_consensus.cc:482] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 22:59:46.883633 11105 raft_consensus.cc:3037] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:46.885830 11105 raft_consensus.cc:504] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:46.886426 11105 leader_election.cc:288] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [CANDIDATE]: Term 1 election: Requested vote from peers f35e4a5b9689458a9d8f7dc96f77914a (127.2.237.2:46657), cec82689d41d4bd5a5a588ccfadd4d9f (127.2.237.3:35123)
I20260620 22:59:46.886945 10890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e0a71c55ea2406da78c7ad7742d96b7" candidate_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35e4a5b9689458a9d8f7dc96f77914a"
I20260620 22:59:46.887014 11025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e0a71c55ea2406da78c7ad7742d96b7" candidate_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f"
I20260620 22:59:46.887143 10890 raft_consensus.cc:3037] T 4e0a71c55ea2406da78c7ad7742d96b7 P f35e4a5b9689458a9d8f7dc96f77914a [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:46.887223 11025 raft_consensus.cc:3037] T 4e0a71c55ea2406da78c7ad7742d96b7 P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:46.889109 10890 raft_consensus.cc:2454] T 4e0a71c55ea2406da78c7ad7742d96b7 P f35e4a5b9689458a9d8f7dc96f77914a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc6e2cea39945e0b6f1c53d5b2b93d6 in term 1.
I20260620 22:59:46.889525 10680 leader_election.cc:302] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc6e2cea39945e0b6f1c53d5b2b93d6, f35e4a5b9689458a9d8f7dc96f77914a; no voters:
I20260620 22:59:46.889804 11105 raft_consensus.cc:2781] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:59:46.890304 11105 raft_consensus.cc:686] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 1 LEADER]: Becoming Leader. State: Replica: 5cc6e2cea39945e0b6f1c53d5b2b93d6, State: Running, Role: LEADER
I20260620 22:59:46.890444 11025 raft_consensus.cc:2454] T 4e0a71c55ea2406da78c7ad7742d96b7 P cec82689d41d4bd5a5a588ccfadd4d9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc6e2cea39945e0b6f1c53d5b2b93d6 in term 1.
I20260620 22:59:46.890751 11105 consensus_queue.cc:237] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:46.894438 10590 catalog_manager.cc:5495] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc6e2cea39945e0b6f1c53d5b2b93d6 (127.2.237.1). New cstate: current_term: 1 leader_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } health_report { overall_health: UNKNOWN } } }
Jun 20 22:59:46 dist-test-slave-f63b krb5kdc[10545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996384, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.62@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM
I20260620 22:59:46.991433 11136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:46.977129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:42559 (local address 127.0.0.1:55774)
0620 22:59:46.977899 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:46.977923 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:46.978090 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:46.978441 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:46.978446 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:46.979223 (+ 777us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:46.979467 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:46.979476 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.980568 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.980575 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:46.981574 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:46.981585 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.982299 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.982305 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:46.982489 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:46.983276 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:46.983293 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:46.985733 (+ 2440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:46.988884 (+ 3151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:46.988897 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:46.988905 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:46.989292 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:46.989672 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:46.989678 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:46.989682 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:46.989834 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:46.990330 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:46.990337 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:46.990501 (+ 164us) client_negotiation.cc:769] Sending connection context
0620 22:59:46.990748 (+ 247us) client_negotiation.cc:240] Negotiation successful
0620 22:59:46.991027 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":420,"threads_started":1}
I20260620 22:59:46.991503 11137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:46.977221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55774 (local address 127.2.237.62:42559)
0620 22:59:46.978167 (+ 946us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:46.978172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:46.978195 (+ 23us) server_negotiation.cc:407] Connection header received
0620 22:59:46.978248 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:46.978251 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:46.978301 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:46.978411 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:46.979642 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.980440 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.981715 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:46.982137 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:46.982274 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:46.985945 (+ 3671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:46.985950 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:46.985953 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:46.985999 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:46.988741 (+ 2742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:46.989432 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:46.989437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:46.989455 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:46.989569 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:46.989937 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:46.989942 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:46.989945 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:46.990191 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:46.990355 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:46.990812 (+ 457us) server_negotiation.cc:299] Negotiation successful
0620 22:59:46.991139 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":139,"server-negotiator.queue_time_us":827,"thread_start_us":621,"threads_started":1}
I20260620 22:59:46.997716 10590 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:55774:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260620 22:59:46.998827 10590 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 22:59:47.006102 10727 tablet_service.cc:1467] Processing CreateTablet for tablet 6a8784981e8b4f9db31a7c69321d5fc9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7d677e32256845dc8b8cfc26821f672d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 22:59:47.006681 10866 tablet_service.cc:1467] Processing CreateTablet for tablet 6a8784981e8b4f9db31a7c69321d5fc9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7d677e32256845dc8b8cfc26821f672d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 22:59:47.007453 10866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a8784981e8b4f9db31a7c69321d5fc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:47.007511 10727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a8784981e8b4f9db31a7c69321d5fc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:47.007450 11005 tablet_service.cc:1467] Processing CreateTablet for tablet 6a8784981e8b4f9db31a7c69321d5fc9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7d677e32256845dc8b8cfc26821f672d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260620 22:59:47.008199 11005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a8784981e8b4f9db31a7c69321d5fc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:47.015077 11102 tablet_bootstrap.cc:492] T 6a8784981e8b4f9db31a7c69321d5fc9 P f35e4a5b9689458a9d8f7dc96f77914a: Bootstrap starting.
I20260620 22:59:47.015470 11101 tablet_bootstrap.cc:492] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f: Bootstrap starting.
I20260620 22:59:47.016463 11099 tablet_bootstrap.cc:492] T 6a8784981e8b4f9db31a7c69321d5fc9 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Bootstrap starting.
I20260620 22:59:47.017071 11102 tablet_bootstrap.cc:654] T 6a8784981e8b4f9db31a7c69321d5fc9 P f35e4a5b9689458a9d8f7dc96f77914a: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:47.018152 11099 tablet_bootstrap.cc:654] T 6a8784981e8b4f9db31a7c69321d5fc9 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:47.018194 11101 tablet_bootstrap.cc:654] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:47.024919 11102 tablet_bootstrap.cc:492] T 6a8784981e8b4f9db31a7c69321d5fc9 P f35e4a5b9689458a9d8f7dc96f77914a: No bootstrap required, opened a new log
I20260620 22:59:47.025182 11102 ts_tablet_manager.cc:1397] T 6a8784981e8b4f9db31a7c69321d5fc9 P f35e4a5b9689458a9d8f7dc96f77914a: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.001s
I20260620 22:59:47.026109 11102 raft_consensus.cc:348] T 6a8784981e8b4f9db31a7c69321d5fc9 P f35e4a5b9689458a9d8f7dc96f77914a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:47.026455 11102 raft_consensus.cc:374] T 6a8784981e8b4f9db31a7c69321d5fc9 P f35e4a5b9689458a9d8f7dc96f77914a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:47.026623 11102 raft_consensus.cc:729] T 6a8784981e8b4f9db31a7c69321d5fc9 P f35e4a5b9689458a9d8f7dc96f77914a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f35e4a5b9689458a9d8f7dc96f77914a, State: Initialized, Role: FOLLOWER
I20260620 22:59:47.027065 11102 consensus_queue.cc:260] T 6a8784981e8b4f9db31a7c69321d5fc9 P f35e4a5b9689458a9d8f7dc96f77914a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:47.027633 11107 tablet_replica.cc:439] T 6a8784981e8b4f9db31a7c69321d5fc9 P f35e4a5b9689458a9d8f7dc96f77914a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } } }
I20260620 22:59:47.028023 11107 tablet_replica.cc:442] T 6a8784981e8b4f9db31a7c69321d5fc9 P f35e4a5b9689458a9d8f7dc96f77914a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 22:59:47.028268 11099 tablet_bootstrap.cc:492] T 6a8784981e8b4f9db31a7c69321d5fc9 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: No bootstrap required, opened a new log
I20260620 22:59:47.028462 11099 ts_tablet_manager.cc:1397] T 6a8784981e8b4f9db31a7c69321d5fc9 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.003s
I20260620 22:59:47.029385 11099 raft_consensus.cc:348] T 6a8784981e8b4f9db31a7c69321d5fc9 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:47.029703 11099 raft_consensus.cc:374] T 6a8784981e8b4f9db31a7c69321d5fc9 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:47.029788 11099 raft_consensus.cc:729] T 6a8784981e8b4f9db31a7c69321d5fc9 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc6e2cea39945e0b6f1c53d5b2b93d6, State: Initialized, Role: FOLLOWER
I20260620 22:59:47.030059 11099 consensus_queue.cc:260] T 6a8784981e8b4f9db31a7c69321d5fc9 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:47.030608 11105 tablet_replica.cc:439] T 6a8784981e8b4f9db31a7c69321d5fc9 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } } }
I20260620 22:59:47.030934 11105 tablet_replica.cc:442] T 6a8784981e8b4f9db31a7c69321d5fc9 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 22:59:47.031519 11099 ts_tablet_manager.cc:1428] T 6a8784981e8b4f9db31a7c69321d5fc9 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260620 22:59:47.033245 11101 tablet_bootstrap.cc:492] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f: No bootstrap required, opened a new log
I20260620 22:59:47.033504 11101 ts_tablet_manager.cc:1397] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f: Time spent bootstrapping tablet: real 0.018s user 0.011s sys 0.001s
I20260620 22:59:47.034437 11101 raft_consensus.cc:348] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:47.034798 11101 raft_consensus.cc:374] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:47.034937 11101 raft_consensus.cc:729] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cec82689d41d4bd5a5a588ccfadd4d9f, State: Initialized, Role: FOLLOWER
I20260620 22:59:47.035109 11102 ts_tablet_manager.cc:1428] T 6a8784981e8b4f9db31a7c69321d5fc9 P f35e4a5b9689458a9d8f7dc96f77914a: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20260620 22:59:47.035465 11101 consensus_queue.cc:260] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:47.035985 11109 tablet_replica.cc:439] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } } }
I20260620 22:59:47.036340 11109 tablet_replica.cc:442] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 22:59:47.037412 11101 ts_tablet_manager.cc:1428] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260620 22:59:47.070557 11109 raft_consensus.cc:482] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 22:59:47.070818 11109 raft_consensus.cc:504] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:47.071509 11109 leader_election.cc:288] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc6e2cea39945e0b6f1c53d5b2b93d6 (127.2.237.1:44539), f35e4a5b9689458a9d8f7dc96f77914a (127.2.237.2:46657)
I20260620 22:59:47.072046 10746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a8784981e8b4f9db31a7c69321d5fc9" candidate_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" is_pre_election: true
I20260620 22:59:47.072150 10890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a8784981e8b4f9db31a7c69321d5fc9" candidate_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" is_pre_election: true
I20260620 22:59:47.072356 10746 raft_consensus.cc:2454] T 6a8784981e8b4f9db31a7c69321d5fc9 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cec82689d41d4bd5a5a588ccfadd4d9f in term 0.
I20260620 22:59:47.072489 10890 raft_consensus.cc:2454] T 6a8784981e8b4f9db31a7c69321d5fc9 P f35e4a5b9689458a9d8f7dc96f77914a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cec82689d41d4bd5a5a588ccfadd4d9f in term 0.
I20260620 22:59:47.072844 10959 leader_election.cc:302] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc6e2cea39945e0b6f1c53d5b2b93d6, cec82689d41d4bd5a5a588ccfadd4d9f; no voters:
I20260620 22:59:47.073195 11109 raft_consensus.cc:2781] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 22:59:47.073299 11109 raft_consensus.cc:482] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 22:59:47.073457 11109 raft_consensus.cc:3037] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:47.075789 11109 raft_consensus.cc:504] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:47.076349 11109 leader_election.cc:288] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f [CANDIDATE]: Term 1 election: Requested vote from peers 5cc6e2cea39945e0b6f1c53d5b2b93d6 (127.2.237.1:44539), f35e4a5b9689458a9d8f7dc96f77914a (127.2.237.2:46657)
I20260620 22:59:47.076820 10746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a8784981e8b4f9db31a7c69321d5fc9" candidate_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6"
I20260620 22:59:47.076897 10890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a8784981e8b4f9db31a7c69321d5fc9" candidate_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35e4a5b9689458a9d8f7dc96f77914a"
I20260620 22:59:47.077024 10746 raft_consensus.cc:3037] T 6a8784981e8b4f9db31a7c69321d5fc9 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:47.077178 10890 raft_consensus.cc:3037] T 6a8784981e8b4f9db31a7c69321d5fc9 P f35e4a5b9689458a9d8f7dc96f77914a [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:47.079629 10746 raft_consensus.cc:2454] T 6a8784981e8b4f9db31a7c69321d5fc9 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cec82689d41d4bd5a5a588ccfadd4d9f in term 1.
I20260620 22:59:47.080073 10959 leader_election.cc:302] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc6e2cea39945e0b6f1c53d5b2b93d6, cec82689d41d4bd5a5a588ccfadd4d9f; no voters:
I20260620 22:59:47.080322 11109 raft_consensus.cc:2781] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:59:47.080413 10890 raft_consensus.cc:2454] T 6a8784981e8b4f9db31a7c69321d5fc9 P f35e4a5b9689458a9d8f7dc96f77914a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cec82689d41d4bd5a5a588ccfadd4d9f in term 1.
I20260620 22:59:47.080528 11109 raft_consensus.cc:686] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f [term 1 LEADER]: Becoming Leader. State: Replica: cec82689d41d4bd5a5a588ccfadd4d9f, State: Running, Role: LEADER
I20260620 22:59:47.081027 11109 consensus_queue.cc:237] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:47.082882 11106 tablet_replica.cc:439] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f: TxnStatusTablet state changed. Reason: New leader cec82689d41d4bd5a5a588ccfadd4d9f. Latest consensus state: current_term: 1 leader_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } } }
I20260620 22:59:47.083223 11106 tablet_replica.cc:442] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f: This TxnStatusTablet replica's current role is: LEADER
I20260620 22:59:47.083803 11142 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260620 22:59:47.084010 11142 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260620 22:59:47.084774 10590 catalog_manager.cc:5495] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f reported cstate change: term changed from 0 to 1, leader changed from <none> to cec82689d41d4bd5a5a588ccfadd4d9f (127.2.237.3). New cstate: current_term: 1 leader_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } health_report { overall_health: HEALTHY } } }
I20260620 22:59:47.107956 11136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:47.102118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:35123 (local address 127.0.0.1:46902)
0620 22:59:47.102377 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:47.102401 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:47.102549 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:47.102982 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:47.102989 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:47.103006 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 22:59:47.103205 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:47.103217 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:47.104180 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:47.104186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:47.105785 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:47.105801 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:47.106660 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:47.106669 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:47.106900 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:47.106988 (+ 88us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 22:59:47.107524 (+ 536us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 22:59:47.107544 (+ 20us) client_negotiation.cc:769] Sending connection context
0620 22:59:47.107622 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 22:59:47.107749 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260620 22:59:47.108072 11097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:47.102119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46902 (local address 127.2.237.3:35123)
0620 22:59:47.102428 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:47.102432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:47.102451 (+ 19us) server_negotiation.cc:407] Connection header received
0620 22:59:47.102775 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:47.102779 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:47.102839 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:47.102916 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 22:59:47.103345 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:47.104047 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:47.105951 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:47.106489 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:47.106639 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:47.107247 (+ 608us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 22:59:47.107402 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 22:59:47.107518 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:47.107666 (+ 148us) server_negotiation.cc:299] Negotiation successful
0620 22:59:47.107755 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202}
I20260620 22:59:47.111858 10890 raft_consensus.cc:1261] T 6a8784981e8b4f9db31a7c69321d5fc9 P f35e4a5b9689458a9d8f7dc96f77914a [term 1 FOLLOWER]: Refusing update from remote peer cec82689d41d4bd5a5a588ccfadd4d9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:59:47.111922 10746 raft_consensus.cc:1261] T 6a8784981e8b4f9db31a7c69321d5fc9 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 1 FOLLOWER]: Refusing update from remote peer cec82689d41d4bd5a5a588ccfadd4d9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:59:47.112432 11109 consensus_queue.cc:1035] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:59:47.112800 11106 consensus_queue.cc:1035] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:59:47.116204 11107 tablet_replica.cc:439] T 6a8784981e8b4f9db31a7c69321d5fc9 P f35e4a5b9689458a9d8f7dc96f77914a: TxnStatusTablet state changed. Reason: New leader cec82689d41d4bd5a5a588ccfadd4d9f. Latest consensus state: current_term: 1 leader_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } } }
I20260620 22:59:47.116523 11107 tablet_replica.cc:442] T 6a8784981e8b4f9db31a7c69321d5fc9 P f35e4a5b9689458a9d8f7dc96f77914a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 22:59:47.118323 11109 tablet_replica.cc:439] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } } }
I20260620 22:59:47.118650 11109 tablet_replica.cc:442] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f: This TxnStatusTablet replica's current role is: LEADER
I20260620 22:59:47.121219 11105 tablet_replica.cc:439] T 6a8784981e8b4f9db31a7c69321d5fc9 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: TxnStatusTablet state changed. Reason: New leader cec82689d41d4bd5a5a588ccfadd4d9f. Latest consensus state: current_term: 1 leader_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } } }
I20260620 22:59:47.121518 11105 tablet_replica.cc:442] T 6a8784981e8b4f9db31a7c69321d5fc9 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 22:59:47.122999 11106 tablet_replica.cc:439] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } } }
I20260620 22:59:47.123260 11106 tablet_replica.cc:442] T 6a8784981e8b4f9db31a7c69321d5fc9 P cec82689d41d4bd5a5a588ccfadd4d9f: This TxnStatusTablet replica's current role is: LEADER
I20260620 22:59:47.124272 11114 mvcc.cc:205] Tried to move back new op lower bound from 7299057201603887104 to 7299057201485828096. Current Snapshot: MvccSnapshot[applied={T|T < 7299057201603887104}]
I20260620 22:59:47.124315 11107 tablet_replica.cc:439] T 6a8784981e8b4f9db31a7c69321d5fc9 P f35e4a5b9689458a9d8f7dc96f77914a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } } }
I20260620 22:59:47.124519 11107 tablet_replica.cc:442] T 6a8784981e8b4f9db31a7c69321d5fc9 P f35e4a5b9689458a9d8f7dc96f77914a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 22:59:47.125461 11105 tablet_replica.cc:439] T 6a8784981e8b4f9db31a7c69321d5fc9 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } } }
I20260620 22:59:47.125744 11105 tablet_replica.cc:442] T 6a8784981e8b4f9db31a7c69321d5fc9 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260620 22:59:47.138177 11146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:47.125895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48372 (local address 127.2.237.1:44539)
0620 22:59:47.130847 (+ 4952us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:47.130854 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:47.130879 (+ 25us) server_negotiation.cc:407] Connection header received
0620 22:59:47.130986 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:47.130991 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:47.131093 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:47.131275 (+ 182us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 22:59:47.131812 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:47.133036 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:47.135107 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:47.135880 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:47.136034 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:47.136783 (+ 749us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 22:59:47.137484 (+ 701us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 22:59:47.137643 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:47.137770 (+ 127us) server_negotiation.cc:299] Negotiation successful
0620 22:59:47.137854 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4811,"thread_start_us":4651,"threads_started":1}
I20260620 22:59:47.158233 11097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:47.151087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:45131 (local address 127.2.237.3:35123)
0620 22:59:47.151312 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:47.151318 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:47.151604 (+ 286us) server_negotiation.cc:407] Connection header received
0620 22:59:47.151811 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:47.151816 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:47.151906 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:47.152000 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 22:59:47.152658 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:47.153800 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:47.156076 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:47.156774 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:47.156893 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:47.157458 (+ 565us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 22:59:47.157622 (+ 164us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 22:59:47.157705 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:47.157858 (+ 153us) server_negotiation.cc:299] Negotiation successful
0620 22:59:47.157934 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260620 22:59:47.158429 11149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:47.150744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:35123 (local address 127.2.237.1:45131)
0620 22:59:47.151469 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:47.151497 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:47.151656 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:47.152034 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:47.152040 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:47.152053 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 22:59:47.152471 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:47.152484 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:47.153950 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:47.153956 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:47.155923 (+ 1967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:47.155938 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:47.156975 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:47.156983 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:47.157187 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:47.157261 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 22:59:47.157727 (+ 466us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 22:59:47.157740 (+ 13us) client_negotiation.cc:769] Sending connection context
0620 22:59:47.157831 (+ 91us) client_negotiation.cc:240] Negotiation successful
0620 22:59:47.157987 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":576,"threads_started":1}
I20260620 22:59:47.172140 11025 raft_consensus.cc:1261] T 4e0a71c55ea2406da78c7ad7742d96b7 P cec82689d41d4bd5a5a588ccfadd4d9f [term 1 FOLLOWER]: Refusing update from remote peer 5cc6e2cea39945e0b6f1c53d5b2b93d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:59:47.173049 10890 raft_consensus.cc:1261] T 4e0a71c55ea2406da78c7ad7742d96b7 P f35e4a5b9689458a9d8f7dc96f77914a [term 1 FOLLOWER]: Refusing update from remote peer 5cc6e2cea39945e0b6f1c53d5b2b93d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:59:47.173096 11129 consensus_queue.cc:1035] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:59:47.173702 11129 consensus_queue.cc:1035] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:59:47.234829 11146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:47.221240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:43853 (local address 127.2.237.1:44539)
0620 22:59:47.221395 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:47.221402 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:47.227941 (+ 6539us) server_negotiation.cc:407] Connection header received
0620 22:59:47.228138 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:47.228142 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:47.228249 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:47.228341 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 22:59:47.229050 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:47.230079 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:47.232641 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:47.233257 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:47.233370 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:47.233903 (+ 533us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 22:59:47.234033 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 22:59:47.234115 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:47.234276 (+ 161us) server_negotiation.cc:299] Negotiation successful
0620 22:59:47.234358 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 22:59:47.234875 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:47.220905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:44539 (local address 127.2.237.3:43853)
0620 22:59:47.227817 (+ 6912us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:47.227838 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:47.227978 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:47.228385 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:47.228391 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:47.228405 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 22:59:47.228872 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:47.228887 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:47.230221 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:47.230227 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:47.232487 (+ 2260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:47.232506 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:47.233468 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:47.233475 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:47.233648 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:47.233724 (+ 76us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 22:59:47.234141 (+ 417us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 22:59:47.234153 (+ 12us) client_negotiation.cc:769] Sending connection context
0620 22:59:47.234241 (+ 88us) client_negotiation.cc:240] Negotiation successful
0620 22:59:47.234390 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6704,"thread_start_us":6585,"threads_started":1}
I20260620 22:59:47.274308 10590 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 22:59:47.274621 10590 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 22:59:47.278065 10590 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cbbbed944e234507bd103f29eeb1325c: Sending DeleteTablet for 3 replicas of tablet 4e0a71c55ea2406da78c7ad7742d96b7
I20260620 22:59:47.278919 10727 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e0a71c55ea2406da78c7ad7742d96b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:47 UTC) from {username='oryx', principal='oryx/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:57960
I20260620 22:59:47.279099 10866 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e0a71c55ea2406da78c7ad7742d96b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:47 UTC) from {username='oryx', principal='oryx/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:60774
I20260620 22:59:47.279654 11005 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e0a71c55ea2406da78c7ad7742d96b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:47 UTC) from {username='oryx', principal='oryx/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:38116
I20260620 22:59:47.288441 11162 tablet_replica.cc:330] T 4e0a71c55ea2406da78c7ad7742d96b7 P cec82689d41d4bd5a5a588ccfadd4d9f: stopping tablet replica
I20260620 22:59:47.288659 11162 raft_consensus.cc:2229] T 4e0a71c55ea2406da78c7ad7742d96b7 P cec82689d41d4bd5a5a588ccfadd4d9f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 22:59:47.288964 11162 raft_consensus.cc:2258] T 4e0a71c55ea2406da78c7ad7742d96b7 P cec82689d41d4bd5a5a588ccfadd4d9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:47.290974 11161 tablet_replica.cc:330] T 4e0a71c55ea2406da78c7ad7742d96b7 P f35e4a5b9689458a9d8f7dc96f77914a: stopping tablet replica
I20260620 22:59:47.291649 11161 raft_consensus.cc:2229] T 4e0a71c55ea2406da78c7ad7742d96b7 P f35e4a5b9689458a9d8f7dc96f77914a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 22:59:47.292227 11161 raft_consensus.cc:2258] T 4e0a71c55ea2406da78c7ad7742d96b7 P f35e4a5b9689458a9d8f7dc96f77914a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:47.293736 11161 ts_tablet_manager.cc:1905] T 4e0a71c55ea2406da78c7ad7742d96b7 P f35e4a5b9689458a9d8f7dc96f77914a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:47.294812 11160 tablet_replica.cc:330] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: stopping tablet replica
I20260620 22:59:47.302242 11161 ts_tablet_manager.cc:1918] T 4e0a71c55ea2406da78c7ad7742d96b7 P f35e4a5b9689458a9d8f7dc96f77914a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 22:59:47.302610 11160 raft_consensus.cc:2229] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 1 LEADER]: Raft consensus shutting down.
I20260620 22:59:47.303269 11160 raft_consensus.cc:2258] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:47.304409 11160 ts_tablet_manager.cc:1905] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:47.310686 11161 log.cc:1196] T 4e0a71c55ea2406da78c7ad7742d96b7 P f35e4a5b9689458a9d8f7dc96f77914a: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-1/wal/wals/4e0a71c55ea2406da78c7ad7742d96b7
I20260620 22:59:47.310932 11162 ts_tablet_manager.cc:1905] T 4e0a71c55ea2406da78c7ad7742d96b7 P cec82689d41d4bd5a5a588ccfadd4d9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:47.311339 11161 ts_tablet_manager.cc:1939] T 4e0a71c55ea2406da78c7ad7742d96b7 P f35e4a5b9689458a9d8f7dc96f77914a: Deleting consensus metadata
I20260620 22:59:47.313189 11160 ts_tablet_manager.cc:1918] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 22:59:47.314520 10577 catalog_manager.cc:4841] TS f35e4a5b9689458a9d8f7dc96f77914a (127.2.237.2:46657): tablet 4e0a71c55ea2406da78c7ad7742d96b7 (table test-table [id=4d2518219de84fdc86bd8e856462835d]) successfully deleted
I20260620 22:59:47.315013 11160 log.cc:1196] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-0/wal/wals/4e0a71c55ea2406da78c7ad7742d96b7
I20260620 22:59:47.315613 11160 ts_tablet_manager.cc:1939] T 4e0a71c55ea2406da78c7ad7742d96b7 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Deleting consensus metadata
I20260620 22:59:47.316722 10577 catalog_manager.cc:4841] TS 5cc6e2cea39945e0b6f1c53d5b2b93d6 (127.2.237.1:44539): tablet 4e0a71c55ea2406da78c7ad7742d96b7 (table test-table [id=4d2518219de84fdc86bd8e856462835d]) successfully deleted
I20260620 22:59:47.318382 11162 ts_tablet_manager.cc:1918] T 4e0a71c55ea2406da78c7ad7742d96b7 P cec82689d41d4bd5a5a588ccfadd4d9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 22:59:47.318542 11162 log.cc:1196] T 4e0a71c55ea2406da78c7ad7742d96b7 P cec82689d41d4bd5a5a588ccfadd4d9f: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-2/wal/wals/4e0a71c55ea2406da78c7ad7742d96b7
I20260620 22:59:47.319669 11162 ts_tablet_manager.cc:1939] T 4e0a71c55ea2406da78c7ad7742d96b7 P cec82689d41d4bd5a5a588ccfadd4d9f: Deleting consensus metadata
I20260620 22:59:47.320775 10575 catalog_manager.cc:4841] TS cec82689d41d4bd5a5a588ccfadd4d9f (127.2.237.3:35123): tablet 4e0a71c55ea2406da78c7ad7742d96b7 (table test-table [id=4d2518219de84fdc86bd8e856462835d]) successfully deleted
Jun 20 22:59:47 dist-test-slave-f63b krb5kdc[10545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 20 22:59:47 dist-test-slave-f63b krb5kdc[10545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 22:59:47 dist-test-slave-f63b krb5kdc[10545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
W20260620 22:59:47.367161 11137 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 22:59:47.354193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55782 (local address 127.2.237.62:42559)
0620 22:59:47.354408 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:47.354415 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:47.355006 (+ 591us) server_negotiation.cc:407] Connection header received
0620 22:59:47.355200 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:47.355207 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:47.355293 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:47.355404 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:47.357072 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:47.358247 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:47.360101 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:47.360852 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:47.360999 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:47.366877 (+ 5878us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55782: BlockingRecv error: recv got EOF from 127.0.0.1:55782 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Jun 20 22:59:47 dist-test-slave-f63b krb5kdc[10545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM
I20260620 22:59:47.397163 11137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:47.375267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55786 (local address 127.2.237.62:42559)
0620 22:59:47.375460 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:47.375467 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:47.378979 (+ 3512us) server_negotiation.cc:407] Connection header received
0620 22:59:47.379240 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:47.379246 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:47.379331 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:47.379442 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:47.381215 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:47.382411 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:47.384343 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:47.384981 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:47.385108 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:47.390092 (+ 4984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:47.390099 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:47.390104 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:47.390150 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:47.393794 (+ 3644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:47.394669 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:47.394675 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:47.394679 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:47.394837 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:47.395308 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:47.395314 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:47.395317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:47.395672 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:47.395936 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:47.396538 (+ 602us) server_negotiation.cc:299] Negotiation successful
0620 22:59:47.396782 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260620 22:59:47.401141 10590 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55786:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 22:59:47.401834 10590 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 22:59:47.410037 11005 tablet_service.cc:1467] Processing CreateTablet for tablet dd92f5762d3d4781adc069f6b4fc6b58 (DEFAULT_TABLE table=test-table [id=8d7862068b744a6b9963d7d5b807ae00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:59:47.410194 10727 tablet_service.cc:1467] Processing CreateTablet for tablet dd92f5762d3d4781adc069f6b4fc6b58 (DEFAULT_TABLE table=test-table [id=8d7862068b744a6b9963d7d5b807ae00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:59:47.410495 11005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd92f5762d3d4781adc069f6b4fc6b58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:47.410641 10727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd92f5762d3d4781adc069f6b4fc6b58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:47.414790 10866 tablet_service.cc:1467] Processing CreateTablet for tablet dd92f5762d3d4781adc069f6b4fc6b58 (DEFAULT_TABLE table=test-table [id=8d7862068b744a6b9963d7d5b807ae00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:59:47.415220 10866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd92f5762d3d4781adc069f6b4fc6b58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:47.417763 11101 tablet_bootstrap.cc:492] T dd92f5762d3d4781adc069f6b4fc6b58 P cec82689d41d4bd5a5a588ccfadd4d9f: Bootstrap starting.
I20260620 22:59:47.419469 11101 tablet_bootstrap.cc:654] T dd92f5762d3d4781adc069f6b4fc6b58 P cec82689d41d4bd5a5a588ccfadd4d9f: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:47.425514 11102 tablet_bootstrap.cc:492] T dd92f5762d3d4781adc069f6b4fc6b58 P f35e4a5b9689458a9d8f7dc96f77914a: Bootstrap starting.
I20260620 22:59:47.427201 11102 tablet_bootstrap.cc:654] T dd92f5762d3d4781adc069f6b4fc6b58 P f35e4a5b9689458a9d8f7dc96f77914a: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:47.430502 11101 tablet_bootstrap.cc:492] T dd92f5762d3d4781adc069f6b4fc6b58 P cec82689d41d4bd5a5a588ccfadd4d9f: No bootstrap required, opened a new log
I20260620 22:59:47.430732 11101 ts_tablet_manager.cc:1397] T dd92f5762d3d4781adc069f6b4fc6b58 P cec82689d41d4bd5a5a588ccfadd4d9f: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.009s
I20260620 22:59:47.430830 11099 tablet_bootstrap.cc:492] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Bootstrap starting.
I20260620 22:59:47.431588 11101 raft_consensus.cc:348] T dd92f5762d3d4781adc069f6b4fc6b58 P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:47.431842 11101 raft_consensus.cc:374] T dd92f5762d3d4781adc069f6b4fc6b58 P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:47.431933 11101 raft_consensus.cc:729] T dd92f5762d3d4781adc069f6b4fc6b58 P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cec82689d41d4bd5a5a588ccfadd4d9f, State: Initialized, Role: FOLLOWER
I20260620 22:59:47.432255 11101 consensus_queue.cc:260] T dd92f5762d3d4781adc069f6b4fc6b58 P cec82689d41d4bd5a5a588ccfadd4d9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:47.432489 11099 tablet_bootstrap.cc:654] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:47.435546 11101 ts_tablet_manager.cc:1428] T dd92f5762d3d4781adc069f6b4fc6b58 P cec82689d41d4bd5a5a588ccfadd4d9f: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260620 22:59:47.439910 11102 tablet_bootstrap.cc:492] T dd92f5762d3d4781adc069f6b4fc6b58 P f35e4a5b9689458a9d8f7dc96f77914a: No bootstrap required, opened a new log
I20260620 22:59:47.440078 11102 ts_tablet_manager.cc:1397] T dd92f5762d3d4781adc069f6b4fc6b58 P f35e4a5b9689458a9d8f7dc96f77914a: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.001s
I20260620 22:59:47.440830 11102 raft_consensus.cc:348] T dd92f5762d3d4781adc069f6b4fc6b58 P f35e4a5b9689458a9d8f7dc96f77914a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:47.441113 11102 raft_consensus.cc:374] T dd92f5762d3d4781adc069f6b4fc6b58 P f35e4a5b9689458a9d8f7dc96f77914a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:47.442440 11102 raft_consensus.cc:729] T dd92f5762d3d4781adc069f6b4fc6b58 P f35e4a5b9689458a9d8f7dc96f77914a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f35e4a5b9689458a9d8f7dc96f77914a, State: Initialized, Role: FOLLOWER
I20260620 22:59:47.442991 11102 consensus_queue.cc:260] T dd92f5762d3d4781adc069f6b4fc6b58 P f35e4a5b9689458a9d8f7dc96f77914a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:47.444054 11102 ts_tablet_manager.cc:1428] T dd92f5762d3d4781adc069f6b4fc6b58 P f35e4a5b9689458a9d8f7dc96f77914a: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260620 22:59:47.451411 11099 tablet_bootstrap.cc:492] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: No bootstrap required, opened a new log
I20260620 22:59:47.451648 11099 ts_tablet_manager.cc:1397] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.006s
I20260620 22:59:47.452528 11099 raft_consensus.cc:348] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:47.452811 11099 raft_consensus.cc:374] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:47.452950 11099 raft_consensus.cc:729] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc6e2cea39945e0b6f1c53d5b2b93d6, State: Initialized, Role: FOLLOWER
I20260620 22:59:47.453204 11099 consensus_queue.cc:260] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:47.453763 11099 ts_tablet_manager.cc:1428] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260620 22:59:47.576085 11105 raft_consensus.cc:482] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 22:59:47.576333 11105 raft_consensus.cc:504] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:47.577091 11105 leader_election.cc:288] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f35e4a5b9689458a9d8f7dc96f77914a (127.2.237.2:46657), cec82689d41d4bd5a5a588ccfadd4d9f (127.2.237.3:35123)
I20260620 22:59:47.577708 10889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd92f5762d3d4781adc069f6b4fc6b58" candidate_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" is_pre_election: true
I20260620 22:59:47.577880 11025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd92f5762d3d4781adc069f6b4fc6b58" candidate_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" is_pre_election: true
I20260620 22:59:47.577950 10889 raft_consensus.cc:2454] T dd92f5762d3d4781adc069f6b4fc6b58 P f35e4a5b9689458a9d8f7dc96f77914a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc6e2cea39945e0b6f1c53d5b2b93d6 in term 0.
I20260620 22:59:47.578135 11025 raft_consensus.cc:2454] T dd92f5762d3d4781adc069f6b4fc6b58 P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cc6e2cea39945e0b6f1c53d5b2b93d6 in term 0.
I20260620 22:59:47.578441 10680 leader_election.cc:302] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc6e2cea39945e0b6f1c53d5b2b93d6, f35e4a5b9689458a9d8f7dc96f77914a; no voters:
I20260620 22:59:47.578913 11105 raft_consensus.cc:2781] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 22:59:47.579025 11105 raft_consensus.cc:482] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 22:59:47.579151 11105 raft_consensus.cc:3037] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:47.582264 11105 raft_consensus.cc:504] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:47.583119 11105 leader_election.cc:288] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [CANDIDATE]: Term 1 election: Requested vote from peers f35e4a5b9689458a9d8f7dc96f77914a (127.2.237.2:46657), cec82689d41d4bd5a5a588ccfadd4d9f (127.2.237.3:35123)
I20260620 22:59:47.583474 10889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd92f5762d3d4781adc069f6b4fc6b58" candidate_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35e4a5b9689458a9d8f7dc96f77914a"
I20260620 22:59:47.583676 10889 raft_consensus.cc:3037] T dd92f5762d3d4781adc069f6b4fc6b58 P f35e4a5b9689458a9d8f7dc96f77914a [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:47.583657 11025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd92f5762d3d4781adc069f6b4fc6b58" candidate_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f"
I20260620 22:59:47.583866 11025 raft_consensus.cc:3037] T dd92f5762d3d4781adc069f6b4fc6b58 P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:47.586670 10889 raft_consensus.cc:2454] T dd92f5762d3d4781adc069f6b4fc6b58 P f35e4a5b9689458a9d8f7dc96f77914a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc6e2cea39945e0b6f1c53d5b2b93d6 in term 1.
I20260620 22:59:47.587175 10680 leader_election.cc:302] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc6e2cea39945e0b6f1c53d5b2b93d6, f35e4a5b9689458a9d8f7dc96f77914a; no voters:
I20260620 22:59:47.587293 11025 raft_consensus.cc:2454] T dd92f5762d3d4781adc069f6b4fc6b58 P cec82689d41d4bd5a5a588ccfadd4d9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cc6e2cea39945e0b6f1c53d5b2b93d6 in term 1.
I20260620 22:59:47.587558 11105 raft_consensus.cc:2781] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:59:47.587777 11105 raft_consensus.cc:686] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 1 LEADER]: Becoming Leader. State: Replica: 5cc6e2cea39945e0b6f1c53d5b2b93d6, State: Running, Role: LEADER
I20260620 22:59:47.588109 11105 consensus_queue.cc:237] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:47.591130 10590 catalog_manager.cc:5495] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cc6e2cea39945e0b6f1c53d5b2b93d6 (127.2.237.1). New cstate: current_term: 1 leader_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } health_report { overall_health: UNKNOWN } } }
I20260620 22:59:47.612578 11146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:47.605540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48388 (local address 127.2.237.1:44539)
0620 22:59:47.605742 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:47.605748 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:47.605946 (+ 198us) server_negotiation.cc:407] Connection header received
0620 22:59:47.606169 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:47.606175 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:47.606293 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:47.606420 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 22:59:47.607091 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:47.608321 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:47.610226 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:47.610935 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:47.611082 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:47.611582 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 22:59:47.611721 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 22:59:47.611817 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:47.611972 (+ 155us) server_negotiation.cc:299] Negotiation successful
0620 22:59:47.612062 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260620 22:59:47.615787 11025 raft_consensus.cc:1261] T dd92f5762d3d4781adc069f6b4fc6b58 P cec82689d41d4bd5a5a588ccfadd4d9f [term 1 FOLLOWER]: Refusing update from remote peer 5cc6e2cea39945e0b6f1c53d5b2b93d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:59:47.615855 10889 raft_consensus.cc:1261] T dd92f5762d3d4781adc069f6b4fc6b58 P f35e4a5b9689458a9d8f7dc96f77914a [term 1 FOLLOWER]: Refusing update from remote peer 5cc6e2cea39945e0b6f1c53d5b2b93d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:59:47.616504 11158 consensus_queue.cc:1035] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:59:47.617144 11156 consensus_queue.cc:1035] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:59:47.639649 10589 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 22:59:47.639886 10589 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 22:59:47.644101 10589 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cbbbed944e234507bd103f29eeb1325c: Sending DeleteTablet for 3 replicas of tablet dd92f5762d3d4781adc069f6b4fc6b58
I20260620 22:59:47.644973 10866 tablet_service.cc:1514] Processing DeleteTablet for tablet dd92f5762d3d4781adc069f6b4fc6b58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:47 UTC) from {username='oryx', principal='oryx/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:60774
I20260620 22:59:47.645076 11005 tablet_service.cc:1514] Processing DeleteTablet for tablet dd92f5762d3d4781adc069f6b4fc6b58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:47 UTC) from {username='oryx', principal='oryx/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:38116
I20260620 22:59:47.645257 11161 tablet_replica.cc:330] T dd92f5762d3d4781adc069f6b4fc6b58 P f35e4a5b9689458a9d8f7dc96f77914a: stopping tablet replica
I20260620 22:59:47.645362 11162 tablet_replica.cc:330] T dd92f5762d3d4781adc069f6b4fc6b58 P cec82689d41d4bd5a5a588ccfadd4d9f: stopping tablet replica
I20260620 22:59:47.645440 11161 raft_consensus.cc:2229] T dd92f5762d3d4781adc069f6b4fc6b58 P f35e4a5b9689458a9d8f7dc96f77914a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 22:59:47.645655 11161 raft_consensus.cc:2258] T dd92f5762d3d4781adc069f6b4fc6b58 P f35e4a5b9689458a9d8f7dc96f77914a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:47.646355 10727 tablet_service.cc:1514] Processing DeleteTablet for tablet dd92f5762d3d4781adc069f6b4fc6b58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:47 UTC) from {username='oryx', principal='oryx/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:57960
I20260620 22:59:47.646662 11160 tablet_replica.cc:330] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: stopping tablet replica
I20260620 22:59:47.646862 11160 raft_consensus.cc:2229] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 1 LEADER]: Raft consensus shutting down.
I20260620 22:59:47.647142 11160 raft_consensus.cc:2258] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:47.647517 10589 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55786:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 22:59:47.648145 10589 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 22:59:47.649181 11160 ts_tablet_manager.cc:1905] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:47.656288 10727 tablet_service.cc:1467] Processing CreateTablet for tablet 5909c74e880546e986030dbaf29fb706 (DEFAULT_TABLE table=test-table [id=c32155ce539c4d9897faf6e70a5bb369]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:59:47.656582 11005 tablet_service.cc:1467] Processing CreateTablet for tablet 5909c74e880546e986030dbaf29fb706 (DEFAULT_TABLE table=test-table [id=c32155ce539c4d9897faf6e70a5bb369]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:59:47.656739 10727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5909c74e880546e986030dbaf29fb706. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:47.657060 11005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5909c74e880546e986030dbaf29fb706. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:47.660095 10866 tablet_service.cc:1467] Processing CreateTablet for tablet 5909c74e880546e986030dbaf29fb706 (DEFAULT_TABLE table=test-table [id=c32155ce539c4d9897faf6e70a5bb369]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 22:59:47.660511 10866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5909c74e880546e986030dbaf29fb706. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:47.664968 11101 tablet_bootstrap.cc:492] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f: Bootstrap starting.
I20260620 22:59:47.666626 11101 tablet_bootstrap.cc:654] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:47.672011 11162 raft_consensus.cc:2229] T dd92f5762d3d4781adc069f6b4fc6b58 P cec82689d41d4bd5a5a588ccfadd4d9f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 22:59:47.673028 11162 pending_rounds.cc:62] T dd92f5762d3d4781adc069f6b4fc6b58 P cec82689d41d4bd5a5a588ccfadd4d9f: Trying to abort 2 pending ops.
I20260620 22:59:47.673168 11162 pending_rounds.cc:68] T dd92f5762d3d4781adc069f6b4fc6b58 P cec82689d41d4bd5a5a588ccfadd4d9f: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7299057203667132416 op_type: WRITE_OP write_request { tablet_id: "dd92f5762d3d4781adc069f6b4fc6b58" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\217\271\334\321\006\"8\n\ntest-admin\022*\n 8d7862068b744a6b9963d7d5b807ae00\020\001\030\001 \001(\001" signature: "<redacted>""\014\"\224\223Sg\370\37293-\007^ \337\244\201\022Z\335\342m\375\210\034Gw\225\333;\241+v\206\037n\264oq\357fMs~\225\301\222\254c\252\177*\277\363G4\242\310K}x#\307s" signing_key_seq_num: 0 } } request_id { client_id: "77d2388db8764797bb689af6ae3cf289" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260620 22:59:47.673804 11162 pending_rounds.cc:68] T dd92f5762d3d4781adc069f6b4fc6b58 P cec82689d41d4bd5a5a588ccfadd4d9f: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7299057203563548672 op_type: NO_OP noop_request { }
I20260620 22:59:47.673913 11162 raft_consensus.cc:2866] T dd92f5762d3d4781adc069f6b4fc6b58 P cec82689d41d4bd5a5a588ccfadd4d9f [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260620 22:59:47.673997 11162 raft_consensus.cc:2258] T dd92f5762d3d4781adc069f6b4fc6b58 P cec82689d41d4bd5a5a588ccfadd4d9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:47.675490 11161 ts_tablet_manager.cc:1905] T dd92f5762d3d4781adc069f6b4fc6b58 P f35e4a5b9689458a9d8f7dc96f77914a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:47.678205 11101 tablet_bootstrap.cc:492] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f: No bootstrap required, opened a new log
I20260620 22:59:47.678354 11101 ts_tablet_manager.cc:1397] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.004s
I20260620 22:59:47.678328 11160 ts_tablet_manager.cc:1918] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 22:59:47.679219 11101 raft_consensus.cc:348] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:47.679469 11101 raft_consensus.cc:374] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:47.679602 11101 raft_consensus.cc:729] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cec82689d41d4bd5a5a588ccfadd4d9f, State: Initialized, Role: FOLLOWER
I20260620 22:59:47.679857 11101 consensus_queue.cc:260] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:47.680428 11101 ts_tablet_manager.cc:1428] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260620 22:59:47.681107 11102 tablet_bootstrap.cc:492] T 5909c74e880546e986030dbaf29fb706 P f35e4a5b9689458a9d8f7dc96f77914a: Bootstrap starting.
W20260620 22:59:47.681068 11181 consensus_queue.cc:1159] T dd92f5762d3d4781adc069f6b4fc6b58 P cec82689d41d4bd5a5a588ccfadd4d9f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
I20260620 22:59:47.681882 11162 ts_tablet_manager.cc:1905] T dd92f5762d3d4781adc069f6b4fc6b58 P cec82689d41d4bd5a5a588ccfadd4d9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:47.682772 11102 tablet_bootstrap.cc:654] T 5909c74e880546e986030dbaf29fb706 P f35e4a5b9689458a9d8f7dc96f77914a: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:47.683813 11099 tablet_bootstrap.cc:492] T 5909c74e880546e986030dbaf29fb706 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Bootstrap starting.
I20260620 22:59:47.685505 11099 tablet_bootstrap.cc:654] T 5909c74e880546e986030dbaf29fb706 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:47.689260 11161 ts_tablet_manager.cc:1918] T dd92f5762d3d4781adc069f6b4fc6b58 P f35e4a5b9689458a9d8f7dc96f77914a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 22:59:47.692171 11162 ts_tablet_manager.cc:1918] T dd92f5762d3d4781adc069f6b4fc6b58 P cec82689d41d4bd5a5a588ccfadd4d9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 22:59:47.692387 11161 log.cc:1196] T dd92f5762d3d4781adc069f6b4fc6b58 P f35e4a5b9689458a9d8f7dc96f77914a: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-1/wal/wals/dd92f5762d3d4781adc069f6b4fc6b58
I20260620 22:59:47.692548 11162 log.cc:1196] T dd92f5762d3d4781adc069f6b4fc6b58 P cec82689d41d4bd5a5a588ccfadd4d9f: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-2/wal/wals/dd92f5762d3d4781adc069f6b4fc6b58
I20260620 22:59:47.693033 11161 ts_tablet_manager.cc:1939] T dd92f5762d3d4781adc069f6b4fc6b58 P f35e4a5b9689458a9d8f7dc96f77914a: Deleting consensus metadata
I20260620 22:59:47.693475 11162 ts_tablet_manager.cc:1939] T dd92f5762d3d4781adc069f6b4fc6b58 P cec82689d41d4bd5a5a588ccfadd4d9f: Deleting consensus metadata
I20260620 22:59:47.694623 11160 log.cc:1196] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-0/wal/wals/dd92f5762d3d4781adc069f6b4fc6b58
I20260620 22:59:47.695082 11160 ts_tablet_manager.cc:1939] T dd92f5762d3d4781adc069f6b4fc6b58 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Deleting consensus metadata
I20260620 22:59:47.695813 10575 catalog_manager.cc:4841] TS cec82689d41d4bd5a5a588ccfadd4d9f (127.2.237.3:35123): tablet dd92f5762d3d4781adc069f6b4fc6b58 (table test-table [id=8d7862068b744a6b9963d7d5b807ae00]) successfully deleted
I20260620 22:59:47.696201 10577 catalog_manager.cc:4841] TS 5cc6e2cea39945e0b6f1c53d5b2b93d6 (127.2.237.1:44539): tablet dd92f5762d3d4781adc069f6b4fc6b58 (table test-table [id=8d7862068b744a6b9963d7d5b807ae00]) successfully deleted
I20260620 22:59:47.697090 10577 catalog_manager.cc:4841] TS f35e4a5b9689458a9d8f7dc96f77914a (127.2.237.2:46657): tablet dd92f5762d3d4781adc069f6b4fc6b58 (table test-table [id=8d7862068b744a6b9963d7d5b807ae00]) successfully deleted
I20260620 22:59:47.703982 11099 tablet_bootstrap.cc:492] T 5909c74e880546e986030dbaf29fb706 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: No bootstrap required, opened a new log
I20260620 22:59:47.704169 11099 ts_tablet_manager.cc:1397] T 5909c74e880546e986030dbaf29fb706 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.009s
I20260620 22:59:47.705044 11099 raft_consensus.cc:348] T 5909c74e880546e986030dbaf29fb706 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:47.705390 11099 raft_consensus.cc:374] T 5909c74e880546e986030dbaf29fb706 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:47.705529 11099 raft_consensus.cc:729] T 5909c74e880546e986030dbaf29fb706 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc6e2cea39945e0b6f1c53d5b2b93d6, State: Initialized, Role: FOLLOWER
I20260620 22:59:47.705827 11099 consensus_queue.cc:260] T 5909c74e880546e986030dbaf29fb706 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:47.706226 11153 raft_consensus.cc:482] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 22:59:47.706383 11153 raft_consensus.cc:504] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:47.707093 11153 leader_election.cc:288] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cc6e2cea39945e0b6f1c53d5b2b93d6 (127.2.237.1:44539), f35e4a5b9689458a9d8f7dc96f77914a (127.2.237.2:46657)
I20260620 22:59:47.707770 10746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5909c74e880546e986030dbaf29fb706" candidate_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" is_pre_election: true
I20260620 22:59:47.707984 10746 raft_consensus.cc:2454] T 5909c74e880546e986030dbaf29fb706 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cec82689d41d4bd5a5a588ccfadd4d9f in term 0.
I20260620 22:59:47.708772 10890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5909c74e880546e986030dbaf29fb706" candidate_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" is_pre_election: true
I20260620 22:59:47.708981 11099 ts_tablet_manager.cc:1428] T 5909c74e880546e986030dbaf29fb706 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Time spent starting tablet: real 0.005s user 0.001s sys 0.002s
W20260620 22:59:47.709698 10959 leader_election.cc:341] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f35e4a5b9689458a9d8f7dc96f77914a (127.2.237.2:46657): Illegal state: must be running to vote when last-logged opid is not known
I20260620 22:59:47.709951 10959 leader_election.cc:302] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5cc6e2cea39945e0b6f1c53d5b2b93d6, cec82689d41d4bd5a5a588ccfadd4d9f; no voters: f35e4a5b9689458a9d8f7dc96f77914a
I20260620 22:59:47.710304 11106 raft_consensus.cc:2781] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 22:59:47.710413 11106 raft_consensus.cc:482] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 22:59:47.710487 11106 raft_consensus.cc:3037] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:47.712258 11102 tablet_bootstrap.cc:492] T 5909c74e880546e986030dbaf29fb706 P f35e4a5b9689458a9d8f7dc96f77914a: No bootstrap required, opened a new log
I20260620 22:59:47.712438 11102 ts_tablet_manager.cc:1397] T 5909c74e880546e986030dbaf29fb706 P f35e4a5b9689458a9d8f7dc96f77914a: Time spent bootstrapping tablet: real 0.031s user 0.005s sys 0.004s
I20260620 22:59:47.713244 11102 raft_consensus.cc:348] T 5909c74e880546e986030dbaf29fb706 P f35e4a5b9689458a9d8f7dc96f77914a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:47.713493 11102 raft_consensus.cc:374] T 5909c74e880546e986030dbaf29fb706 P f35e4a5b9689458a9d8f7dc96f77914a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:47.713581 11102 raft_consensus.cc:729] T 5909c74e880546e986030dbaf29fb706 P f35e4a5b9689458a9d8f7dc96f77914a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f35e4a5b9689458a9d8f7dc96f77914a, State: Initialized, Role: FOLLOWER
I20260620 22:59:47.713685 11106 raft_consensus.cc:504] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:47.713862 11102 consensus_queue.cc:260] T 5909c74e880546e986030dbaf29fb706 P f35e4a5b9689458a9d8f7dc96f77914a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:47.714890 10746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5909c74e880546e986030dbaf29fb706" candidate_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6"
I20260620 22:59:47.715097 10746 raft_consensus.cc:3037] T 5909c74e880546e986030dbaf29fb706 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:47.716632 11102 ts_tablet_manager.cc:1428] T 5909c74e880546e986030dbaf29fb706 P f35e4a5b9689458a9d8f7dc96f77914a: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260620 22:59:47.717144 11106 leader_election.cc:288] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f [CANDIDATE]: Term 1 election: Requested vote from peers 5cc6e2cea39945e0b6f1c53d5b2b93d6 (127.2.237.1:44539), f35e4a5b9689458a9d8f7dc96f77914a (127.2.237.2:46657)
I20260620 22:59:47.718004 10890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5909c74e880546e986030dbaf29fb706" candidate_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35e4a5b9689458a9d8f7dc96f77914a"
I20260620 22:59:47.718123 10746 raft_consensus.cc:2454] T 5909c74e880546e986030dbaf29fb706 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cec82689d41d4bd5a5a588ccfadd4d9f in term 1.
I20260620 22:59:47.718231 10890 raft_consensus.cc:3037] T 5909c74e880546e986030dbaf29fb706 P f35e4a5b9689458a9d8f7dc96f77914a [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:47.718647 10959 leader_election.cc:302] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cc6e2cea39945e0b6f1c53d5b2b93d6, cec82689d41d4bd5a5a588ccfadd4d9f; no voters:
I20260620 22:59:47.718961 11106 raft_consensus.cc:2781] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:59:47.719408 11106 raft_consensus.cc:686] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f [term 1 LEADER]: Becoming Leader. State: Replica: cec82689d41d4bd5a5a588ccfadd4d9f, State: Running, Role: LEADER
I20260620 22:59:47.719681 11106 consensus_queue.cc:237] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } }
I20260620 22:59:47.721031 10890 raft_consensus.cc:2454] T 5909c74e880546e986030dbaf29fb706 P f35e4a5b9689458a9d8f7dc96f77914a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cec82689d41d4bd5a5a588ccfadd4d9f in term 1.
I20260620 22:59:47.722218 10589 catalog_manager.cc:5495] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f reported cstate change: term changed from 0 to 1, leader changed from <none> to cec82689d41d4bd5a5a588ccfadd4d9f (127.2.237.3). New cstate: current_term: 1 leader_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cec82689d41d4bd5a5a588ccfadd4d9f" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 35123 } health_report { overall_health: HEALTHY } } }
I20260620 22:59:47.743949 11187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:47.737754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46908 (local address 127.2.237.3:35123)
0620 22:59:47.738917 (+ 1163us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:47.738923 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:47.738952 (+ 29us) server_negotiation.cc:407] Connection header received
0620 22:59:47.739014 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:47.739019 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:47.739097 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:47.739197 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 22:59:47.739543 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:47.740666 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:47.741993 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:47.742633 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:47.742752 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:47.743202 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 22:59:47.743324 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 22:59:47.743414 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:47.743539 (+ 125us) server_negotiation.cc:299] Negotiation successful
0620 22:59:47.743615 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1059,"thread_start_us":960,"threads_started":1}
I20260620 22:59:47.757279 11187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:47.751568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:44851 (local address 127.2.237.3:35123)
0620 22:59:47.751742 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:47.751750 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:47.752230 (+ 480us) server_negotiation.cc:407] Connection header received
0620 22:59:47.752360 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:47.752365 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:47.752446 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:47.752537 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 22:59:47.752920 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:47.753980 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:47.755371 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:47.755955 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:47.756065 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:47.756419 (+ 354us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 22:59:47.756533 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 22:59:47.756614 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:47.756875 (+ 261us) server_negotiation.cc:299] Negotiation successful
0620 22:59:47.756958 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":51}
I20260620 22:59:47.757328 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:47.751590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:35123 (local address 127.2.237.3:44851)
0620 22:59:47.752138 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:47.752154 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:47.752253 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:47.752553 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:47.752556 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:47.752565 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0620 22:59:47.752792 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:47.752799 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:47.754102 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:47.754105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:47.755246 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:47.755257 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:47.756131 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:47.756136 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:47.756249 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:47.756282 (+ 33us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0620 22:59:47.756718 (+ 436us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0620 22:59:47.756728 (+ 10us) client_negotiation.cc:769] Sending connection context
0620 22:59:47.756857 (+ 129us) client_negotiation.cc:240] Negotiation successful
0620 22:59:47.756998 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":310,"threads_started":1}
I20260620 22:59:47.765110 10746 raft_consensus.cc:1261] T 5909c74e880546e986030dbaf29fb706 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 1 FOLLOWER]: Refusing update from remote peer cec82689d41d4bd5a5a588ccfadd4d9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:59:47.765164 10889 raft_consensus.cc:1261] T 5909c74e880546e986030dbaf29fb706 P f35e4a5b9689458a9d8f7dc96f77914a [term 1 FOLLOWER]: Refusing update from remote peer cec82689d41d4bd5a5a588ccfadd4d9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 22:59:47.765746 11186 consensus_queue.cc:1035] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cc6e2cea39945e0b6f1c53d5b2b93d6" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 44539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:59:47.766064 11153 consensus_queue.cc:1035] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f35e4a5b9689458a9d8f7dc96f77914a" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 22:59:47.817230 10589 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 22:59:47.817438 10589 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 22:59:47.820250 10589 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cbbbed944e234507bd103f29eeb1325c: Sending DeleteTablet for 3 replicas of tablet 5909c74e880546e986030dbaf29fb706
I20260620 22:59:47.820971 10727 tablet_service.cc:1514] Processing DeleteTablet for tablet 5909c74e880546e986030dbaf29fb706 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:47 UTC) from {username='oryx', principal='oryx/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:57960
I20260620 22:59:47.821030 10866 tablet_service.cc:1514] Processing DeleteTablet for tablet 5909c74e880546e986030dbaf29fb706 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:47 UTC) from {username='oryx', principal='oryx/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:60774
I20260620 22:59:47.821201 11004 tablet_service.cc:1514] Processing DeleteTablet for tablet 5909c74e880546e986030dbaf29fb706 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 22:59:47 UTC) from {username='oryx', principal='oryx/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:38116
I20260620 22:59:47.821286 11160 tablet_replica.cc:330] T 5909c74e880546e986030dbaf29fb706 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: stopping tablet replica
I20260620 22:59:47.821472 11162 tablet_replica.cc:330] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f: stopping tablet replica
I20260620 22:59:47.821460 11160 raft_consensus.cc:2229] T 5909c74e880546e986030dbaf29fb706 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 22:59:47.821661 11160 raft_consensus.cc:2258] T 5909c74e880546e986030dbaf29fb706 P 5cc6e2cea39945e0b6f1c53d5b2b93d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:47.821660 11162 raft_consensus.cc:2229] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f [term 1 LEADER]: Raft consensus shutting down.
I20260620 22:59:47.821971 11161 tablet_replica.cc:330] T 5909c74e880546e986030dbaf29fb706 P f35e4a5b9689458a9d8f7dc96f77914a: stopping tablet replica
I20260620 22:59:47.821979 11162 raft_consensus.cc:2258] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:47.822176 11161 raft_consensus.cc:2229] T 5909c74e880546e986030dbaf29fb706 P f35e4a5b9689458a9d8f7dc96f77914a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 22:59:47.822389 11161 raft_consensus.cc:2258] T 5909c74e880546e986030dbaf29fb706 P f35e4a5b9689458a9d8f7dc96f77914a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 22:59:47.822626 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 10661
I20260620 22:59:47.823091 11162 ts_tablet_manager.cc:1905] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:47.824090 11161 ts_tablet_manager.cc:1905] T 5909c74e880546e986030dbaf29fb706 P f35e4a5b9689458a9d8f7dc96f77914a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:47.824177 11160 ts_tablet_manager.cc:1905] T 5909c74e880546e986030dbaf29fb706 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 22:59:47.829665 11162 ts_tablet_manager.cc:1918] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 22:59:47.829785 11162 log.cc:1196] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-2/wal/wals/5909c74e880546e986030dbaf29fb706
I20260620 22:59:47.830188 11162 ts_tablet_manager.cc:1939] T 5909c74e880546e986030dbaf29fb706 P cec82689d41d4bd5a5a588ccfadd4d9f: Deleting consensus metadata
I20260620 22:59:47.830992 11161 ts_tablet_manager.cc:1918] T 5909c74e880546e986030dbaf29fb706 P f35e4a5b9689458a9d8f7dc96f77914a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 22:59:47.831127 11161 log.cc:1196] T 5909c74e880546e986030dbaf29fb706 P f35e4a5b9689458a9d8f7dc96f77914a: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-1/wal/wals/5909c74e880546e986030dbaf29fb706
I20260620 22:59:47.831352 10575 catalog_manager.cc:4841] TS cec82689d41d4bd5a5a588ccfadd4d9f (127.2.237.3:35123): tablet 5909c74e880546e986030dbaf29fb706 (table test-table [id=c32155ce539c4d9897faf6e70a5bb369]) successfully deleted
I20260620 22:59:47.831557 11161 ts_tablet_manager.cc:1939] T 5909c74e880546e986030dbaf29fb706 P f35e4a5b9689458a9d8f7dc96f77914a: Deleting consensus metadata
I20260620 22:59:47.832659 10577 catalog_manager.cc:4841] TS f35e4a5b9689458a9d8f7dc96f77914a (127.2.237.2:46657): tablet 5909c74e880546e986030dbaf29fb706 (table test-table [id=c32155ce539c4d9897faf6e70a5bb369]) successfully deleted
I20260620 22:59:47.832973 11160 ts_tablet_manager.cc:1918] T 5909c74e880546e986030dbaf29fb706 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260620 22:59:47.833115 11160 log.cc:1196] T 5909c74e880546e986030dbaf29fb706 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781996318807807-2996-0/minicluster-data/ts-0/wal/wals/5909c74e880546e986030dbaf29fb706
I20260620 22:59:47.833479 11160 ts_tablet_manager.cc:1939] T 5909c74e880546e986030dbaf29fb706 P 5cc6e2cea39945e0b6f1c53d5b2b93d6: Deleting consensus metadata
I20260620 22:59:47.834386 10577 catalog_manager.cc:4841] TS 5cc6e2cea39945e0b6f1c53d5b2b93d6 (127.2.237.1:44539): tablet 5909c74e880546e986030dbaf29fb706 (table test-table [id=c32155ce539c4d9897faf6e70a5bb369]) successfully deleted
Jun 20 22:59:47 dist-test-slave-f63b krb5kdc[10545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.237.1@KRBTEST.COM
I20260620 22:59:47.846006 11146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:47.823477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48400 (local address 127.2.237.1:44539)
0620 22:59:47.823773 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:47.823779 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:47.832325 (+ 8546us) server_negotiation.cc:407] Connection header received
0620 22:59:47.832598 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:47.832604 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:47.832703 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:47.832788 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:47.834213 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:47.835063 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:47.836240 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:47.836613 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:47.836702 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:47.840404 (+ 3702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:47.840417 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:47.840433 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:47.840462 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:47.843290 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:47.844068 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:47.844076 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:47.844079 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:47.844230 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:47.844654 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:47.844658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:47.844660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:47.845007 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:47.845227 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:47.845457 (+ 230us) server_negotiation.cc:299] Negotiation successful
0620 22:59:47.845691 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168}
I20260620 22:59:47.846683 10792 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:47.970148 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 10661
I20260620 22:59:47.992740 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 10801
Jun 20 22:59:47 dist-test-slave-f63b krb5kdc[10545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.237.2@KRBTEST.COM
I20260620 22:59:48.008435 11196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:47.993565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42732 (local address 127.2.237.2:46657)
0620 22:59:47.994063 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:47.994068 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:47.994084 (+ 16us) server_negotiation.cc:407] Connection header received
0620 22:59:47.994137 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:47.994140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:47.994197 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:47.994294 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:47.995498 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:47.996241 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:47.997423 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:47.997930 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:47.998037 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:48.002136 (+ 4099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:48.002153 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:48.002169 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:48.002206 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:48.005169 (+ 2963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:48.006023 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:48.006032 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:48.006034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:48.006159 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:48.006551 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:48.006558 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:48.006562 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:48.007092 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:48.007457 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:48.007693 (+ 236us) server_negotiation.cc:299] Negotiation successful
0620 22:59:48.007952 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":281,"threads_started":1}
I20260620 22:59:48.009443 10931 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:48.107908 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 10801
I20260620 22:59:48.125658 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 10940
Jun 20 22:59:48 dist-test-slave-f63b krb5kdc[10545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.237.3@KRBTEST.COM
I20260620 22:59:48.139954 11187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:48.126528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46914 (local address 127.2.237.3:35123)
0620 22:59:48.126749 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:48.126755 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:48.126776 (+ 21us) server_negotiation.cc:407] Connection header received
0620 22:59:48.126853 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:48.126857 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:48.126925 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:48.127043 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:48.128299 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:48.129034 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:48.130234 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:48.130772 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:48.130899 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:48.135049 (+ 4150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:48.135064 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:48.135080 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:48.135114 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:48.137682 (+ 2568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:48.138292 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:48.138298 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:48.138301 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:48.138424 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:48.138780 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:48.138783 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:48.138785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:48.139099 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:48.139317 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:48.139521 (+ 204us) server_negotiation.cc:299] Negotiation successful
0620 22:59:48.139686 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260620 22:59:48.140617 11070 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:48.286034 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 10940
I20260620 22:59:48.312470 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 10558
I20260620 22:59:48.313750 10630 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:48.428613 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 10558
2026-06-20T22:59:48Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (5651 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 22:59:48 dist-test-slave-f63b krb5kdc[11202](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 22:59:48 dist-test-slave-f63b krb5kdc[11202](info): set up 2 sockets
Jun 20 22:59:48 dist-test-slave-f63b krb5kdc[11202](info): commencing operation
krb5kdc: starting...
W20260620 22:59:49.543658 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 22:59:49 dist-test-slave-f63b krb5kdc[11202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T22:59:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T22:59:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
I20260620 22:59:49.760279 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:39809
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:36827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:39809
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:50.096822 11215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:50.097174 11215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:50.097342 11215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:50.104578 11215 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 22:59:50.104720 11215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:50.104839 11215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:50.104940 11215 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 22:59:50.105041 11215 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 22:59:50.115446 11215 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:36827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:39809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:39809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:50.116815 11215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:50.118130 11215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:50.130143 11223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:50.130123 11221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:50.130736 11220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:50.131006 11215 server_base.cc:1034] running on GCE node
I20260620 22:59:50.131780 11215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:50.133291 11215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:50.134482 11215 hybrid_clock.cc:648] HybridClock initialized: now 1781996390134431 us; error 61 us; skew 500 ppm
I20260620 22:59:50.134872 11215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-20T22:59:50Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1778 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 22:59:50 dist-test-slave-f63b krb5kdc[11229](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 22:59:50 dist-test-slave-f63b krb5kdc[11229](info): set up 2 sockets
Jun 20 22:59:50 dist-test-slave-f63b krb5kdc[11229](info): commencing operation
krb5kdc: starting...
W20260620 22:59:51.323422 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 22:59:51 dist-test-slave-f63b krb5kdc[11229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T22:59:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T22:59:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
I20260620 22:59:51.542260 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:37983
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:34901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:37983
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:51.860764 11242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:51.861109 11242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:51.861271 11242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:51.868629 11242 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 22:59:51.868777 11242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:51.868886 11242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:51.868985 11242 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 22:59:51.869072 11242 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 22:59:51.878770 11242 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:34901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:37983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:37983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:51.880153 11242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:51.881474 11242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:51.893015 11248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:51.893411 11250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:51.893388 11247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:51.894520 11242 server_base.cc:1034] running on GCE node
I20260620 22:59:51.895305 11242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:51.896744 11242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:51.897969 11242 hybrid_clock.cc:648] HybridClock initialized: now 1781996391897922 us; error 39 us; skew 500 ppm
I20260620 22:59:51.898311 11242 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:51 dist-test-slave-f63b krb5kdc[11229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996391, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:51.901453 11242 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 22:59:51.902930 11242 webserver.cc:460] Webserver started at http://127.2.237.62:33449/ using document root <none> and password file <none>
I20260620 22:59:51.903631 11242 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:51.903774 11242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:51.904075 11242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:51.907065 11242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "ab603c5bb6874a22ba9d3536a1818171"
format_stamp: "Formatted at 2026-06-20 22:59:51 on dist-test-slave-f63b"
server_key: "12e654219ed32b691d92a34212c47b1b"
server_key_iv: "a3bc61d1a347e545554df4ca28a91c23"
server_key_version: "encryptionkey@0"
I20260620 22:59:51.907888 11242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "ab603c5bb6874a22ba9d3536a1818171"
format_stamp: "Formatted at 2026-06-20 22:59:51 on dist-test-slave-f63b"
server_key: "12e654219ed32b691d92a34212c47b1b"
server_key_iv: "a3bc61d1a347e545554df4ca28a91c23"
server_key_version: "encryptionkey@0"
I20260620 22:59:51.913028 11242 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260620 22:59:51.916528 11257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:51.917888 11242 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 22:59:51.918116 11242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "ab603c5bb6874a22ba9d3536a1818171"
format_stamp: "Formatted at 2026-06-20 22:59:51 on dist-test-slave-f63b"
server_key: "12e654219ed32b691d92a34212c47b1b"
server_key_iv: "a3bc61d1a347e545554df4ca28a91c23"
server_key_version: "encryptionkey@0"
I20260620 22:59:51.918313 11242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:51.943387 11242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:51.947834 11242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:51.948141 11242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:51.967245 11242 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:37983
I20260620 22:59:51.967280 11308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:37983 every 8 connection(s)
I20260620 22:59:51.968665 11242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 22:59:51.972921 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 11242
I20260620 22:59:51.973131 11309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 22:59:51.973186 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 22:59:51.974143 2996 external_mini_cluster.cc:1428] Setting key 38cc7e0bb4f9014337b8896838ee5131
Jun 20 22:59:51 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:59:51.990705 11309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171: Bootstrap starting.
I20260620 22:59:51.995929 11309 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171: Neither blocks nor log segments found. Creating new log.
I20260620 22:59:51.997507 11309 log.cc:824] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171: Log is configured to *not* fsync() on all Append() calls
I20260620 22:59:52.000584 11312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:51.976443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60598 (local address 127.2.237.62:37983)
0620 22:59:51.977328 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:51.977343 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:51.977386 (+ 43us) server_negotiation.cc:407] Connection header received
0620 22:59:51.979051 (+ 1665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:51.979073 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:51.979481 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:51.979826 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:51.982059 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:51.983477 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:51.985564 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:51.986280 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:51.986413 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:51.991769 (+ 5356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:51.991781 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:51.991802 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:51.991856 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:51.996324 (+ 4468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:51.997094 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:51.997105 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:51.997114 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:51.997376 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:51.997797 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:51.997801 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:51.997803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:51.998359 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:51.998736 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:51.999201 (+ 465us) server_negotiation.cc:299] Negotiation successful
0620 22:59:51.999606 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":460,"threads_started":1}
I20260620 22:59:52.001235 11309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171: No bootstrap required, opened a new log
I20260620 22:59:52.010251 11309 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab603c5bb6874a22ba9d3536a1818171" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37983 } }
I20260620 22:59:52.010776 11309 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 22:59:52.010950 11309 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab603c5bb6874a22ba9d3536a1818171, State: Initialized, Role: FOLLOWER
I20260620 22:59:52.011785 11309 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab603c5bb6874a22ba9d3536a1818171" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37983 } }
I20260620 22:59:52.012113 11309 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 22:59:52.012300 11309 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 22:59:52.012573 11309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [term 0 FOLLOWER]: Advancing to term 1
I20260620 22:59:52.015260 11309 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab603c5bb6874a22ba9d3536a1818171" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37983 } }
I20260620 22:59:52.015827 11309 leader_election.cc:302] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab603c5bb6874a22ba9d3536a1818171; no voters:
I20260620 22:59:52.016474 11309 leader_election.cc:288] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 22:59:52.016599 11314 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 22:59:52.017414 11314 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [term 1 LEADER]: Becoming Leader. State: Replica: ab603c5bb6874a22ba9d3536a1818171, State: Running, Role: LEADER
I20260620 22:59:52.017977 11314 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab603c5bb6874a22ba9d3536a1818171" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37983 } }
I20260620 22:59:52.019062 11309 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 22:59:52.021030 11315 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab603c5bb6874a22ba9d3536a1818171" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab603c5bb6874a22ba9d3536a1818171" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37983 } } }
I20260620 22:59:52.021077 11316 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab603c5bb6874a22ba9d3536a1818171. Latest consensus state: current_term: 1 leader_uuid: "ab603c5bb6874a22ba9d3536a1818171" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab603c5bb6874a22ba9d3536a1818171" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37983 } } }
I20260620 22:59:52.021538 11316 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:52.021538 11315 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:52.022634 11320 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 22:59:52.028293 11320 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 22:59:52.037436 11320 catalog_manager.cc:1347] Generated new cluster ID: 82472aaed3e64becb36a5a9534bbeaf8
I20260620 22:59:52.037549 11320 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 22:59:52.047943 11320 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 22:59:52.049291 11320 catalog_manager.cc:1504] Loading token signing keys...
I20260620 22:59:52.057638 11320 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171: Generated new TSK 0
I20260620 22:59:52.058462 11320 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.237.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
I20260620 22:59:52.121693 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:0
--local_ip_for_outbound_sockets=127.2.237.1
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:37983
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:34901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:52.450932 11335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:52.451210 11335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:52.451303 11335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:52.458670 11335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:52.458775 11335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:52.458885 11335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 22:59:52.468361 11335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:34901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:37983
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:52.469580 11335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:52.470883 11335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:52.484236 11343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:52.484402 11341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:52.484236 11340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:52.485353 11335 server_base.cc:1034] running on GCE node
I20260620 22:59:52.485931 11335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:52.486892 11335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:52.488099 11335 hybrid_clock.cc:648] HybridClock initialized: now 1781996392488036 us; error 44 us; skew 500 ppm
I20260620 22:59:52.488518 11335 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:52 dist-test-slave-f63b krb5kdc[11229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996392, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:52.491546 11335 init.cc:377] Logged in from keytab as kudu/127.2.237.1@KRBTEST.COM (short username kudu)
I20260620 22:59:52.492875 11335 webserver.cc:460] Webserver started at http://127.2.237.1:37901/ using document root <none> and password file <none>
I20260620 22:59:52.493544 11335 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:52.493671 11335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:52.494015 11335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:52.497006 11335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/data/instance:
uuid: "2a1c03e07c3643668b225733eb7502ca"
format_stamp: "Formatted at 2026-06-20 22:59:52 on dist-test-slave-f63b"
server_key: "9ecf76f327d96b7b850b3e0420ecc87c"
server_key_iv: "30025efde63c48c16e70a172d942ed31"
server_key_version: "encryptionkey@0"
I20260620 22:59:52.497740 11335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance:
uuid: "2a1c03e07c3643668b225733eb7502ca"
format_stamp: "Formatted at 2026-06-20 22:59:52 on dist-test-slave-f63b"
server_key: "9ecf76f327d96b7b850b3e0420ecc87c"
server_key_iv: "30025efde63c48c16e70a172d942ed31"
server_key_version: "encryptionkey@0"
I20260620 22:59:52.502707 11335 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260620 22:59:52.505859 11350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:52.507151 11335 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 22:59:52.507311 11335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "2a1c03e07c3643668b225733eb7502ca"
format_stamp: "Formatted at 2026-06-20 22:59:52 on dist-test-slave-f63b"
server_key: "9ecf76f327d96b7b850b3e0420ecc87c"
server_key_iv: "30025efde63c48c16e70a172d942ed31"
server_key_version: "encryptionkey@0"
I20260620 22:59:52.507503 11335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:52.533401 11335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:52.537541 11335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:52.537858 11335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:52.539062 11335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:59:52.540294 11335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:52.540362 11335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:52.540441 11335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:52.540504 11335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:52.572428 11335 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:45659
I20260620 22:59:52.572481 11462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:45659 every 8 connection(s)
I20260620 22:59:52.573734 11335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 22:59:52.576241 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 11335
I20260620 22:59:52.576419 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance
I20260620 22:59:52.577221 2996 external_mini_cluster.cc:1428] Setting key b4e55cd90df34151af21142e0ac6e256
Jun 20 22:59:52 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996392, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:59:52.595703 11465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:52.576470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:39813 (local address 127.2.237.62:37983)
0620 22:59:52.576999 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:52.577005 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:52.578087 (+ 1082us) server_negotiation.cc:407] Connection header received
0620 22:59:52.579990 (+ 1903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:52.579998 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:52.580085 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:52.580214 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:52.581651 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:52.582821 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:52.584303 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:52.585035 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:52.585170 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:52.588814 (+ 3644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:52.588823 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:52.588831 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:52.588874 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:52.592384 (+ 3510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:52.593092 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:52.593100 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:52.593106 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:52.593249 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:52.593672 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:52.593679 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:52.593685 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:52.594051 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:52.594203 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:52.594928 (+ 725us) server_negotiation.cc:299] Negotiation successful
0620 22:59:52.595184 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":240,"threads_started":1}
I20260620 22:59:52.596356 11466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:52.577052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.1:39813)
0620 22:59:52.577924 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:52.577978 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:52.579721 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:52.580410 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:52.580418 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:52.580926 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:52.581483 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:52.581499 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:52.582945 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:52.582948 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:52.584160 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:52.584171 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:52.585235 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:52.585239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:52.585424 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:52.586157 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:52.586178 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:52.588600 (+ 2422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:52.592543 (+ 3943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:52.592550 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:52.592563 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:52.592976 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:52.593353 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:52.593356 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:52.593358 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:52.593566 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:52.594179 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:52.594184 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:52.594496 (+ 312us) client_negotiation.cc:769] Sending connection context
0620 22:59:52.594799 (+ 303us) client_negotiation.cc:240] Negotiation successful
0620 22:59:52.595177 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":392,"threads_started":1}
I20260620 22:59:52.598069 11463 heartbeater.cc:347] Connected to a master server at 127.2.237.62:37983
I20260620 22:59:52.598421 11463 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:52.599365 11463 heartbeater.cc:511] Master 127.2.237.62:37983 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.2.237.2@KRBTEST.COM; defaulting to no policy
I20260620 22:59:52.601655 11274 ts_manager.cc:194] Registered new tserver with Master: 2a1c03e07c3643668b225733eb7502ca (127.2.237.1:45659)
I20260620 22:59:52.603149 11274 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.1@KRBTEST.COM'} at 127.2.237.1:39813
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
I20260620 22:59:52.628139 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:0
--local_ip_for_outbound_sockets=127.2.237.2
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:37983
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:34901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:52.951858 11469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:52.952165 11469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:52.952265 11469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:52.959416 11469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:52.959532 11469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:52.959648 11469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 22:59:52.969368 11469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:34901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:37983
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:52.970763 11469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:52.972046 11469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:52.985252 11474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:52.985260 11475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:52.985294 11477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:52.986202 11469 server_base.cc:1034] running on GCE node
I20260620 22:59:52.986965 11469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:52.987934 11469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:52.989161 11469 hybrid_clock.cc:648] HybridClock initialized: now 1781996392989116 us; error 62 us; skew 500 ppm
I20260620 22:59:52.989475 11469 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:52 dist-test-slave-f63b krb5kdc[11229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996392, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:52.992651 11469 init.cc:377] Logged in from keytab as kudu/127.2.237.2@KRBTEST.COM (short username kudu)
I20260620 22:59:52.993965 11469 webserver.cc:460] Webserver started at http://127.2.237.2:42569/ using document root <none> and password file <none>
I20260620 22:59:52.994691 11469 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:52.994793 11469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:52.995064 11469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:52.997992 11469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/data/instance:
uuid: "f58b74becd744651897b46b4696f13cc"
format_stamp: "Formatted at 2026-06-20 22:59:52 on dist-test-slave-f63b"
server_key: "225df8240b586ca6d49a386cc41442a4"
server_key_iv: "bde8e3aa8f25b10f08ab24096f18a8ba"
server_key_version: "encryptionkey@0"
I20260620 22:59:52.998756 11469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance:
uuid: "f58b74becd744651897b46b4696f13cc"
format_stamp: "Formatted at 2026-06-20 22:59:52 on dist-test-slave-f63b"
server_key: "225df8240b586ca6d49a386cc41442a4"
server_key_iv: "bde8e3aa8f25b10f08ab24096f18a8ba"
server_key_version: "encryptionkey@0"
I20260620 22:59:53.003680 11469 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.002s
I20260620 22:59:53.007256 11484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:53.008612 11469 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 22:59:53.008795 11469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "f58b74becd744651897b46b4696f13cc"
format_stamp: "Formatted at 2026-06-20 22:59:52 on dist-test-slave-f63b"
server_key: "225df8240b586ca6d49a386cc41442a4"
server_key_iv: "bde8e3aa8f25b10f08ab24096f18a8ba"
server_key_version: "encryptionkey@0"
I20260620 22:59:53.009001 11469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:53.025939 11469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:53.030483 11469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:53.030769 11469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:53.031718 11469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:59:53.033005 11469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:53.033075 11469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:53.033214 11469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:53.033270 11469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:53.065843 11469 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:44543
I20260620 22:59:53.065886 11596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:44543 every 8 connection(s)
I20260620 22:59:53.067210 11469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 22:59:53.068575 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 11469
I20260620 22:59:53.068753 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance
I20260620 22:59:53.069185 2996 external_mini_cluster.cc:1428] Setting key 0877d20e2172468cfeb01246ee3e688e
Jun 20 22:59:53 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996392, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:59:53.088680 11465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:53.069749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:53013 (local address 127.2.237.62:37983)
0620 22:59:53.069959 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:53.069964 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:53.071446 (+ 1482us) server_negotiation.cc:407] Connection header received
0620 22:59:53.073236 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:53.073241 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:53.073332 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:53.073439 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:53.075035 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:53.075817 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:53.077897 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:53.078318 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:53.078413 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:53.082487 (+ 4074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:53.082491 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:53.082495 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:53.082525 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:53.085038 (+ 2513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:53.085955 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:53.085962 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:53.085965 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:53.086177 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:53.086752 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:53.086758 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:53.086762 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:53.087075 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:53.087252 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:53.088023 (+ 771us) server_negotiation.cc:299] Negotiation successful
0620 22:59:53.088281 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260620 22:59:53.089676 11599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:53.070368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.2:53013)
0620 22:59:53.071306 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:53.071344 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:53.072981 (+ 1637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:53.073691 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:53.073700 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:53.074217 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:53.074872 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:53.074891 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:53.075968 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:53.075973 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:53.077723 (+ 1750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:53.077739 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:53.078541 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:53.078550 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:53.078908 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:53.079603 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:53.079635 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:53.082262 (+ 2627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:53.085191 (+ 2929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:53.085200 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:53.085217 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:53.085666 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:53.086319 (+ 653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:53.086325 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:53.086330 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:53.086554 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:53.087227 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:53.087236 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:53.087613 (+ 377us) client_negotiation.cc:769] Sending connection context
0620 22:59:53.087919 (+ 306us) client_negotiation.cc:240] Negotiation successful
0620 22:59:53.088456 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":445,"threads_started":1}
WARNING: no policy specified for kudu/127.2.237.3@KRBTEST.COM; defaulting to no policy
I20260620 22:59:53.091287 11597 heartbeater.cc:347] Connected to a master server at 127.2.237.62:37983
I20260620 22:59:53.091634 11597 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:53.092522 11597 heartbeater.cc:511] Master 127.2.237.62:37983 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.3@KRBTEST.COM" created.
I20260620 22:59:53.094254 11274 ts_manager.cc:194] Registered new tserver with Master: f58b74becd744651897b46b4696f13cc (127.2.237.2:44543)
I20260620 22:59:53.095207 11274 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.2@KRBTEST.COM'} at 127.2.237.2:53013
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
I20260620 22:59:53.117851 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:0
--local_ip_for_outbound_sockets=127.2.237.3
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:37983
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:34901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:53.446836 11602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:53.447182 11602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:53.447360 11602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:53.454807 11602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:53.454972 11602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:53.455152 11602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 22:59:53.465174 11602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:34901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:37983
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:53.466614 11602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:53.468021 11602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:53.481411 11610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:53.481377 11607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:53.481725 11608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:53.482344 11602 server_base.cc:1034] running on GCE node
I20260620 22:59:53.483088 11602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:53.484046 11602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:53.485286 11602 hybrid_clock.cc:648] HybridClock initialized: now 1781996393485239 us; error 45 us; skew 500 ppm
I20260620 22:59:53.485617 11602 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:53 dist-test-slave-f63b krb5kdc[11229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996393, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:53.488869 11602 init.cc:377] Logged in from keytab as kudu/127.2.237.3@KRBTEST.COM (short username kudu)
I20260620 22:59:53.490293 11602 webserver.cc:460] Webserver started at http://127.2.237.3:43577/ using document root <none> and password file <none>
I20260620 22:59:53.491025 11602 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:53.491163 11602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:53.491487 11602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 22:59:53.494390 11602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/data/instance:
uuid: "cd41562ea174454291d82356b4858d47"
format_stamp: "Formatted at 2026-06-20 22:59:53 on dist-test-slave-f63b"
server_key: "980d136be49c00ebfcf28941f24f1c4c"
server_key_iv: "6313705dfc548f265edc8239b064bffe"
server_key_version: "encryptionkey@0"
I20260620 22:59:53.495287 11602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance:
uuid: "cd41562ea174454291d82356b4858d47"
format_stamp: "Formatted at 2026-06-20 22:59:53 on dist-test-slave-f63b"
server_key: "980d136be49c00ebfcf28941f24f1c4c"
server_key_iv: "6313705dfc548f265edc8239b064bffe"
server_key_version: "encryptionkey@0"
I20260620 22:59:53.500217 11602 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260620 22:59:53.503557 11617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:53.504809 11602 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 22:59:53.504957 11602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "cd41562ea174454291d82356b4858d47"
format_stamp: "Formatted at 2026-06-20 22:59:53 on dist-test-slave-f63b"
server_key: "980d136be49c00ebfcf28941f24f1c4c"
server_key_iv: "6313705dfc548f265edc8239b064bffe"
server_key_version: "encryptionkey@0"
I20260620 22:59:53.505163 11602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:53.528270 11602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:53.532245 11602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:53.532536 11602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:53.533504 11602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:59:53.534808 11602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:53.534880 11602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:53.535015 11602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:53.535074 11602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:53.567662 11602 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:45241
I20260620 22:59:53.567713 11729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:45241 every 8 connection(s)
I20260620 22:59:53.568967 11602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 22:59:53.569916 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 11602
I20260620 22:59:53.570088 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance
I20260620 22:59:53.570647 2996 external_mini_cluster.cc:1428] Setting key b2273941ceb62ac1d6d8a36bd8653666
Jun 20 22:59:53 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996393, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 22:59:53.589725 11465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:53.571808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:37541 (local address 127.2.237.62:37983)
0620 22:59:53.572023 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:53.572031 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:53.573581 (+ 1550us) server_negotiation.cc:407] Connection header received
0620 22:59:53.576023 (+ 2442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:53.576027 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:53.576112 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:53.576199 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:53.578062 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:53.578866 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:53.580551 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:53.580944 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:53.581040 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:53.584296 (+ 3256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:53.584301 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:53.584304 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:53.584333 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:53.587258 (+ 2925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:53.587907 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:53.587911 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:53.587914 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:53.588039 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:53.588405 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:53.588410 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:53.588413 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:53.588655 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:53.588762 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:53.589348 (+ 586us) server_negotiation.cc:299] Negotiation successful
0620 22:59:53.589494 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260620 22:59:53.590718 11732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:53.572535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.3:37541)
0620 22:59:53.573437 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:53.573482 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:53.575758 (+ 2276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:53.576463 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:53.576475 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:53.577144 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:53.577887 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:53.577909 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:53.578994 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:53.578997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:53.580328 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:53.580343 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:53.581121 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:53.581125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:53.581311 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:53.581991 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:53.582012 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:53.584073 (+ 2061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 22:59:53.587371 (+ 3298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:53.587377 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:53.587389 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:53.587785 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:53.588150 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 22:59:53.588153 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 22:59:53.588155 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 22:59:53.588314 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 22:59:53.588759 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 22:59:53.588764 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 22:59:53.589039 (+ 275us) client_negotiation.cc:769] Sending connection context
0620 22:59:53.589269 (+ 230us) client_negotiation.cc:240] Negotiation successful
0620 22:59:53.589602 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":363,"threads_started":1}
I20260620 22:59:53.592074 11730 heartbeater.cc:347] Connected to a master server at 127.2.237.62:37983
I20260620 22:59:53.592344 11730 heartbeater.cc:464] Registering TS with master...
I20260620 22:59:53.593050 11730 heartbeater.cc:511] Master 127.2.237.62:37983 requested a full tablet report, sending...
I20260620 22:59:53.594436 11274 ts_manager.cc:194] Registered new tserver with Master: cd41562ea174454291d82356b4858d47 (127.2.237.3:45241)
I20260620 22:59:53.595242 11274 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.3@KRBTEST.COM'} at 127.2.237.3:37541
I20260620 22:59:53.598328 2996 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 22:59:53.606560 11463 heartbeater.cc:503] Master 127.2.237.62:37983 was elected leader, sending a full tablet report...
WARNING: no policy specified for oryx/127.2.237.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.237.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.237.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.1.keytab.
Entry for principal oryx/127.2.237.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.1.keytab.
WARNING: no policy specified for oryx/127.2.237.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.237.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.237.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.2.keytab.
Entry for principal oryx/127.2.237.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.2.keytab.
WARNING: no policy specified for oryx/127.2.237.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.237.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.237.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.3.keytab.
Entry for principal oryx/127.2.237.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.3.keytab.
I20260620 22:59:53.733723 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 11335
Jun 20 22:59:53 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.1@KRBTEST.COM
I20260620 22:59:53.749727 11740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:53.734866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53946 (local address 127.2.237.1:45659)
0620 22:59:53.735684 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:53.735691 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:53.735716 (+ 25us) server_negotiation.cc:407] Connection header received
0620 22:59:53.735775 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:53.735781 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:53.735995 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:53.736139 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:53.737359 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:53.738352 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:53.739634 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:53.740186 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:53.740285 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:53.744498 (+ 4213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:53.744508 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:53.744522 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:53.744559 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:53.747375 (+ 2816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:53.747977 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:53.747984 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:53.747986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:53.748097 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:53.748392 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:53.748395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:53.748397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:53.748706 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:53.748921 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:53.749145 (+ 224us) server_negotiation.cc:299] Negotiation successful
0620 22:59:53.749319 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":482,"threads_started":1}
I20260620 22:59:53.751071 11458 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:53.833750 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 11335
I20260620 22:59:53.850214 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 11469
Jun 20 22:59:53 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.2@KRBTEST.COM
I20260620 22:59:53.865679 11742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:53.851130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34980 (local address 127.2.237.2:44543)
0620 22:59:53.851614 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:53.851620 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:53.851644 (+ 24us) server_negotiation.cc:407] Connection header received
0620 22:59:53.851703 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:53.851709 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:53.851916 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:53.852093 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:53.853302 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:53.854288 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:53.855724 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:53.856283 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:53.856418 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:53.860621 (+ 4203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:53.860631 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:53.860645 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:53.860679 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:53.863382 (+ 2703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:53.863973 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:53.863979 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:53.863982 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:53.864099 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:53.864420 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:53.864423 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:53.864426 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:53.864770 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:53.865009 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:53.865210 (+ 201us) server_negotiation.cc:299] Negotiation successful
0620 22:59:53.865396 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":236,"threads_started":1}
I20260620 22:59:53.866971 11592 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:53.949599 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 11469
I20260620 22:59:53.965348 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 11602
Jun 20 22:59:53 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.3@KRBTEST.COM
I20260620 22:59:53.983268 11744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:53.966375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32774 (local address 127.2.237.3:45241)
0620 22:59:53.967139 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:53.967146 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:53.967176 (+ 30us) server_negotiation.cc:407] Connection header received
0620 22:59:53.967234 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:53.967240 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:53.967459 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:53.967594 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:53.969193 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:53.970239 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:53.972173 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:53.972761 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:53.972863 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:53.977872 (+ 5009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:53.977883 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:53.977897 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:53.977943 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:53.980618 (+ 2675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:53.981214 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:53.981221 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:53.981223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:53.981348 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:53.981649 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:53.981652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:53.981654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:53.982243 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:53.982527 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:53.982757 (+ 230us) server_negotiation.cc:299] Negotiation successful
0620 22:59:53.982985 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":448,"threads_started":1}
I20260620 22:59:53.984614 11725 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:54.067234 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 11602
I20260620 22:59:54.083081 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 11242
I20260620 22:59:54.084304 11304 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 22:59:54.185176 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 11242
I20260620 22:59:54.205636 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:37983
--webserver_interface=127.2.237.62
--webserver_port=33449
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:34901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:37983
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 22:59:54.530836 11747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:54.531186 11747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:54.531342 11747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:54.538720 11747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 22:59:54.538861 11747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:54.538975 11747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:54.539081 11747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 22:59:54.539157 11747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 22:59:54.548996 11747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:34901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:37983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:37983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=33449
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:54.550334 11747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:54.551700 11747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:54.563424 11752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:54.563395 11753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:54.563961 11755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:54.564218 11747 server_base.cc:1034] running on GCE node
I20260620 22:59:54.565026 11747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:54.566332 11747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:54.567600 11747 hybrid_clock.cc:648] HybridClock initialized: now 1781996394567524 us; error 82 us; skew 500 ppm
I20260620 22:59:54.567966 11747 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:54 dist-test-slave-f63b krb5kdc[11229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996394, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:54.571085 11747 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 22:59:54.572746 11747 webserver.cc:460] Webserver started at http://127.2.237.62:33449/ using document root <none> and password file <none>
I20260620 22:59:54.573415 11747 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:54.573575 11747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:54.578919 11747 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260620 22:59:54.581578 11762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:54.582926 11747 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 22:59:54.583102 11747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "ab603c5bb6874a22ba9d3536a1818171"
format_stamp: "Formatted at 2026-06-20 22:59:51 on dist-test-slave-f63b"
server_key: "12e654219ed32b691d92a34212c47b1b"
server_key_iv: "a3bc61d1a347e545554df4ca28a91c23"
server_key_version: "encryptionkey@0"
I20260620 22:59:54.584070 11747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:54.612463 11747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:54.616796 11747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:54.617100 11747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:54.635856 11747 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:37983
I20260620 22:59:54.635887 11813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:37983 every 8 connection(s)
I20260620 22:59:54.637290 11747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 22:59:54.643941 11814 sys_catalog.cc:263] Verifying existing consensus state
I20260620 22:59:54.646150 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 11747
I20260620 22:59:54.647094 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:45659
--local_ip_for_outbound_sockets=127.2.237.1
--tserver_master_addrs=127.2.237.62:37983
--webserver_port=37901
--webserver_interface=127.2.237.1
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:34901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.2.237.1
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 22:59:54.647989 11814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171: Bootstrap starting.
I20260620 22:59:54.668519 11814 log.cc:824] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171: Log is configured to *not* fsync() on all Append() calls
I20260620 22:59:54.677155 11814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 22:59:54.677660 11814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171: Bootstrap complete.
I20260620 22:59:54.686553 11814 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab603c5bb6874a22ba9d3536a1818171" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37983 } }
I20260620 22:59:54.687486 11814 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab603c5bb6874a22ba9d3536a1818171, State: Initialized, Role: FOLLOWER
I20260620 22:59:54.688063 11814 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab603c5bb6874a22ba9d3536a1818171" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37983 } }
I20260620 22:59:54.688361 11814 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 22:59:54.688503 11814 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 22:59:54.688702 11814 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [term 1 FOLLOWER]: Advancing to term 2
I20260620 22:59:54.692416 11814 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab603c5bb6874a22ba9d3536a1818171" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37983 } }
I20260620 22:59:54.692909 11814 leader_election.cc:302] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab603c5bb6874a22ba9d3536a1818171; no voters:
I20260620 22:59:54.693557 11814 leader_election.cc:288] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 22:59:54.693717 11818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [term 2 FOLLOWER]: Leader election won for term 2
I20260620 22:59:54.695978 11814 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 22:59:54.698890 11818 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [term 2 LEADER]: Becoming Leader. State: Replica: ab603c5bb6874a22ba9d3536a1818171, State: Running, Role: LEADER
I20260620 22:59:54.699652 11818 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab603c5bb6874a22ba9d3536a1818171" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37983 } }
I20260620 22:59:54.712697 11819 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ab603c5bb6874a22ba9d3536a1818171" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab603c5bb6874a22ba9d3536a1818171" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37983 } } }
I20260620 22:59:54.713100 11819 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:54.712986 11820 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab603c5bb6874a22ba9d3536a1818171. Latest consensus state: current_term: 2 leader_uuid: "ab603c5bb6874a22ba9d3536a1818171" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab603c5bb6874a22ba9d3536a1818171" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37983 } } }
I20260620 22:59:54.713411 11820 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171 [sys.catalog]: This master's current role is: LEADER
I20260620 22:59:54.714376 11829 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 22:59:54.720306 11829 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 22:59:54.721758 11829 catalog_manager.cc:1259] Loaded cluster ID: 82472aaed3e64becb36a5a9534bbeaf8
I20260620 22:59:54.721915 11829 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 22:59:54.725859 11829 catalog_manager.cc:1504] Loading token signing keys...
I20260620 22:59:54.727361 11829 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ab603c5bb6874a22ba9d3536a1818171: Loaded TSK: 0
I20260620 22:59:54.729152 11829 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 22:59:54.990522 11816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:54.990892 11816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:54.991045 11816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:54.999241 11816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:54.999356 11816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:54.999532 11816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 22:59:55.009742 11816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:34901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.1.keytab
--principal=oryx/127.2.237.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:45659
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=37901
--tserver_master_addrs=127.2.237.62:37983
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:55.011207 11816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:55.012543 11816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:55.026350 11841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:55.026434 11842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:55.026870 11844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:55.027582 11816 server_base.cc:1034] running on GCE node
I20260620 22:59:55.028297 11816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:55.029357 11816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:55.030622 11816 hybrid_clock.cc:648] HybridClock initialized: now 1781996395030511 us; error 66 us; skew 500 ppm
I20260620 22:59:55.030970 11816 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:55 dist-test-slave-f63b krb5kdc[11229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996395, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:55.034255 11816 init.cc:377] Logged in from keytab as oryx/127.2.237.1@KRBTEST.COM (short username oryx)
I20260620 22:59:55.035938 11816 webserver.cc:460] Webserver started at http://127.2.237.1:37901/ using document root <none> and password file <none>
I20260620 22:59:55.036684 11816 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:55.036840 11816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:55.042459 11816 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260620 22:59:55.045298 11851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:55.046800 11816 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 22:59:55.046979 11816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "2a1c03e07c3643668b225733eb7502ca"
format_stamp: "Formatted at 2026-06-20 22:59:52 on dist-test-slave-f63b"
server_key: "9ecf76f327d96b7b850b3e0420ecc87c"
server_key_iv: "30025efde63c48c16e70a172d942ed31"
server_key_version: "encryptionkey@0"
I20260620 22:59:55.047968 11816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:55.069438 11816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:55.073818 11816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:55.074101 11816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:55.075186 11816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:59:55.076557 11816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:55.076700 11816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:55.076825 11816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:55.076913 11816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:55.109866 11816 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:45659
I20260620 22:59:55.109957 11963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:45659 every 8 connection(s)
I20260620 22:59:55.111428 11816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 22:59:55.118381 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 11816
I20260620 22:59:55.119163 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:44543
--local_ip_for_outbound_sockets=127.2.237.2
--tserver_master_addrs=127.2.237.62:37983
--webserver_port=42569
--webserver_interface=127.2.237.2
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:34901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.2.237.2
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 22:59:55 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 22:59:55 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 22:59:55.131034 11967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:55.114471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.1:57813)
0620 22:59:55.115338 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:55.115384 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:55.117023 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:55.118668 (+ 1645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:55.118679 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:55.119282 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:55.119861 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:55.119888 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:55.122306 (+ 2418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:55.122310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:55.123621 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:55.123633 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:55.125085 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:55.125090 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:55.125327 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:55.126085 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:55.126119 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:55.129700 (+ 3581us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":366,"threads_started":1}
W20260620 22:59:55.131062 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 22:59:55.114374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:57813 (local address 127.2.237.62:37983)
0620 22:59:55.115159 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:55.115176 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:55.115507 (+ 331us) server_negotiation.cc:407] Connection header received
0620 22:59:55.117566 (+ 2059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:55.117579 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:55.118025 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:55.118447 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:55.120067 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:55.122139 (+ 2072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:55.123763 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:55.124868 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:55.125052 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:55.130061 (+ 5009us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.1:57813: BlockingRecv error: recv got EOF from 127.2.237.1:57813 (error 108)
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":289,"threads_started":1}
W20260620 22:59:55.131984 11964 heartbeater.cc:644] Failed to heartbeat to 127.2.237.62:37983 (0 consecutive failures): Not authorized: Failed to ping master at 127.2.237.62:37983: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Jun 20 22:59:55 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 22:59:55 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 22:59:55.141005 11967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:55.132613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.1:37487)
0620 22:59:55.132804 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:55.132820 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:55.132898 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:55.133211 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:55.133214 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:55.133499 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:55.133634 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:55.133642 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:55.135186 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:55.135190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:55.136142 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:55.136153 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:55.137206 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:55.137210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:55.137325 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:55.137888 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:55.137904 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:55.140807 (+ 2903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":57}
W20260620 22:59:55.141525 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 22:59:55.132703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:37487 (local address 127.2.237.62:37983)
0620 22:59:55.132914 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:55.132922 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:55.132941 (+ 19us) server_negotiation.cc:407] Connection header received
0620 22:59:55.132997 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:55.133004 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:55.133084 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:55.133187 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:55.133775 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:55.135057 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:55.136292 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:55.137069 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:55.137234 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:55.141084 (+ 3850us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.1:37487: BlockingRecv error: recv got EOF from 127.2.237.1:37487 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Jun 20 22:59:55 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 22:59:55 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 22:59:55.153393 11967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:55.141680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.1:48795)
0620 22:59:55.141969 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:55.141991 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:55.142096 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:55.142417 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:55.142425 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:55.142886 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:55.143095 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:55.143110 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:55.144068 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:55.144074 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:55.145530 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:55.145544 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:55.146235 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:55.146241 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:55.146453 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:55.149880 (+ 3427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:55.149902 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:55.153098 (+ 3196us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156}
W20260620 22:59:55.153469 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 22:59:55.141916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:48795 (local address 127.2.237.62:37983)
0620 22:59:55.142067 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:55.142071 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:55.142085 (+ 14us) server_negotiation.cc:407] Connection header received
0620 22:59:55.142209 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:55.142212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:55.142275 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:55.142349 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:55.143241 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:55.143941 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:55.145651 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:55.146101 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:55.146191 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:55.153261 (+ 7070us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.1:48795: BlockingRecv error: recv got EOF from 127.2.237.1:48795 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260620 22:59:55.153753 11964 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 22:59:55.447965 11968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:55.448256 11968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:55.448419 11968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:55.455814 11968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:55.455916 11968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:55.456120 11968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 22:59:55.465649 11968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:34901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.2.keytab
--principal=oryx/127.2.237.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:44543
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=42569
--tserver_master_addrs=127.2.237.62:37983
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:55.466987 11968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:55.468297 11968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:55.482659 11976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:55.482594 11974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:55.482633 11973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:55.483381 11968 server_base.cc:1034] running on GCE node
I20260620 22:59:55.483933 11968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:55.484779 11968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:55.485985 11968 hybrid_clock.cc:648] HybridClock initialized: now 1781996395485934 us; error 47 us; skew 500 ppm
I20260620 22:59:55.486330 11968 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:55 dist-test-slave-f63b krb5kdc[11229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996395, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:55.489532 11968 init.cc:377] Logged in from keytab as oryx/127.2.237.2@KRBTEST.COM (short username oryx)
I20260620 22:59:55.491063 11968 webserver.cc:460] Webserver started at http://127.2.237.2:42569/ using document root <none> and password file <none>
I20260620 22:59:55.491746 11968 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:55.491897 11968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:55.497104 11968 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260620 22:59:55.499646 11983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:55.500911 11968 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 22:59:55.501085 11968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "f58b74becd744651897b46b4696f13cc"
format_stamp: "Formatted at 2026-06-20 22:59:52 on dist-test-slave-f63b"
server_key: "225df8240b586ca6d49a386cc41442a4"
server_key_iv: "bde8e3aa8f25b10f08ab24096f18a8ba"
server_key_version: "encryptionkey@0"
I20260620 22:59:55.502013 11968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:55.527702 11968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:55.531680 11968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:55.531945 11968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:55.532949 11968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:59:55.534242 11968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:55.534369 11968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:55.534489 11968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:55.534602 11968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:55.566841 11968 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:44543
I20260620 22:59:55.566923 12095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:44543 every 8 connection(s)
I20260620 22:59:55.568312 11968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 22:59:55.573792 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 11968
I20260620 22:59:55.574568 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:45241
--local_ip_for_outbound_sockets=127.2.237.3
--tserver_master_addrs=127.2.237.62:37983
--webserver_port=43577
--webserver_interface=127.2.237.3
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:34901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.2.237.3
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 22:59:55 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 22:59:55 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
W20260620 22:59:55.585469 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 22:59:55.570822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:39203 (local address 127.2.237.62:37983)
0620 22:59:55.571029 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:55.571035 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:55.572234 (+ 1199us) server_negotiation.cc:407] Connection header received
0620 22:59:55.574285 (+ 2051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:55.574289 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:55.574354 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:55.574440 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:55.576542 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:55.577241 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:55.579702 (+ 2461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:55.580091 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:55.580184 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:55.585170 (+ 4986us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.2:39203: BlockingRecv error: recv got EOF from 127.2.237.2:39203 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260620 22:59:55.585944 12098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:55.571395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.2:39203)
0620 22:59:55.572076 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:55.572114 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:55.573978 (+ 1864us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:55.574758 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:55.574773 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:55.575546 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:55.576353 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:55.576377 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:55.577416 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:55.577425 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:55.579554 (+ 2129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:55.579572 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:55.580311 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:55.580321 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:55.580659 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:55.581385 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:55.581419 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:55.584930 (+ 3511us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":254,"threads_started":1}
W20260620 22:59:55.586674 12096 heartbeater.cc:644] Failed to heartbeat to 127.2.237.62:37983 (0 consecutive failures): Not authorized: Failed to ping master at 127.2.237.62:37983: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Jun 20 22:59:55 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 22:59:55 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
W20260620 22:59:55.595623 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 22:59:55.587339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:47749 (local address 127.2.237.62:37983)
0620 22:59:55.587531 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:55.587539 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:55.587558 (+ 19us) server_negotiation.cc:407] Connection header received
0620 22:59:55.587611 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:55.587619 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:55.587694 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:55.587798 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:55.588412 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:55.589608 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:55.590763 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:55.591513 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:55.591664 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:55.595446 (+ 3782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.2:47749: BlockingRecv error: recv got EOF from 127.2.237.2:47749 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 22:59:55.595657 12098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:55.587266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.2:47749)
0620 22:59:55.587433 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:55.587453 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:55.587540 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:55.587805 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:55.587808 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:55.588075 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:55.588287 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:55.588294 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:55.589734 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:55.589738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:55.590642 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:55.590653 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:55.591673 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:55.591678 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:55.591795 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:55.592380 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:55.592396 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:55.595284 (+ 2888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":50}
Jun 20 22:59:55 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 22:59:55 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 22:59:55.604300 12098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:55.596450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.2:54459)
0620 22:59:55.596713 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:55.596728 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:55.596800 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:55.597209 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:55.597214 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:55.597476 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:55.597617 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:55.597625 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:55.599098 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:55.599103 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:55.599970 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:55.599980 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:55.600838 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:55.600842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:55.600954 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:55.601500 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:55.601516 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:55.604096 (+ 2580us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157}
W20260620 22:59:55.604621 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 22:59:55.596599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:54459 (local address 127.2.237.62:37983)
0620 22:59:55.596803 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:55.596810 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:55.596859 (+ 49us) server_negotiation.cc:407] Connection header received
0620 22:59:55.596941 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:55.596949 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:55.597033 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:55.597132 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:55.597751 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:55.598980 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:55.600091 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:55.600711 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:55.600818 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:55.604352 (+ 3534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.2:54459: BlockingRecv error: recv got EOF from 127.2.237.2:54459 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260620 22:59:55.604658 12096 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 22:59:55.897533 12099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 22:59:55.897799 12099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 22:59:55.897888 12099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 22:59:55.904989 12099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 22:59:55.905133 12099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 22:59:55.905316 12099 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 22:59:55.915190 12099 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:34901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/krb5kdc/oryx_127.2.237.3.keytab
--principal=oryx/127.2.237.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:45241
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=43577
--tserver_master_addrs=127.2.237.62:37983
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 22:59:55.916464 12099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 22:59:55.917685 12099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 22:59:55.930631 12104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:55.930783 12107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 22:59:55.930667 12105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 22:59:55.931854 12099 server_base.cc:1034] running on GCE node
I20260620 22:59:55.932381 12099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 22:59:55.933254 12099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 22:59:55.934443 12099 hybrid_clock.cc:648] HybridClock initialized: now 1781996395934390 us; error 53 us; skew 500 ppm
I20260620 22:59:55.934787 12099 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 22:59:55 dist-test-slave-f63b krb5kdc[11229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996395, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.237.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 22:59:55.937887 12099 init.cc:377] Logged in from keytab as oryx/127.2.237.3@KRBTEST.COM (short username oryx)
I20260620 22:59:55.939364 12099 webserver.cc:460] Webserver started at http://127.2.237.3:43577/ using document root <none> and password file <none>
I20260620 22:59:55.940047 12099 fs_manager.cc:359] Metadata directory not provided
I20260620 22:59:55.940196 12099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 22:59:55.945473 12099 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260620 22:59:55.948132 12114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:55.949417 12099 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 22:59:55.949592 12099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "cd41562ea174454291d82356b4858d47"
format_stamp: "Formatted at 2026-06-20 22:59:53 on dist-test-slave-f63b"
server_key: "980d136be49c00ebfcf28941f24f1c4c"
server_key_iv: "6313705dfc548f265edc8239b064bffe"
server_key_version: "encryptionkey@0"
I20260620 22:59:55.950529 12099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 22:59:55.967551 12099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 22:59:55.971964 12099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 22:59:55.972254 12099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 22:59:55.973260 12099 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 22:59:55.974557 12099 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 22:59:55.974728 12099 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:55.974858 12099 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 22:59:55.974946 12099 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 22:59:56.008147 12099 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:45241
I20260620 22:59:56.008248 12226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:45241 every 8 connection(s)
I20260620 22:59:56.009588 12099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 22:59:56.018558 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 12099
Jun 20 22:59:56 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 22:59:56 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
W20260620 22:59:56.026329 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 22:59:56.012204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:34101 (local address 127.2.237.62:37983)
0620 22:59:56.012388 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:56.012392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:56.013526 (+ 1134us) server_negotiation.cc:407] Connection header received
0620 22:59:56.015535 (+ 2009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:56.015540 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:56.015637 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:56.015747 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:56.017265 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:56.017968 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:56.019795 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:56.020503 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:56.020639 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:56.026032 (+ 5393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.3:34101: BlockingRecv error: recv got EOF from 127.2.237.3:34101 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260620 22:59:56.026978 12229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:56.012601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.3:34101)
0620 22:59:56.013340 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:56.013392 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:56.015254 (+ 1862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:56.015998 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:56.016007 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:56.016541 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:56.017092 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:56.017108 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:56.018130 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:56.018136 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:56.019619 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:56.019634 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:56.020744 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:56.020754 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:56.021015 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:56.021713 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:56.021735 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:56.025767 (+ 4032us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":274,"threads_started":1}
W20260620 22:59:56.027729 12227 heartbeater.cc:644] Failed to heartbeat to 127.2.237.62:37983 (0 consecutive failures): Not authorized: Failed to ping master at 127.2.237.62:37983: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Jun 20 22:59:56 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 22:59:56 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 22:59:56.036976 12231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:56.019593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60602 (local address 127.2.237.62:37983)
0620 22:59:56.020381 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:56.020415 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:56.020437 (+ 22us) server_negotiation.cc:407] Connection header received
0620 22:59:56.020486 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:56.020493 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:56.020564 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:56.020663 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:56.022904 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:56.023706 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:56.025563 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:56.026189 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:56.026322 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:56.029569 (+ 3247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 22:59:56.029598 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 22:59:56.029633 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 22:59:56.029676 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0620 22:59:56.033477 (+ 3801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:56.034278 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:56.034284 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:56.034291 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:56.034460 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 22:59:56.034914 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 22:59:56.034919 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 22:59:56.034923 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 22:59:56.035409 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 22:59:56.035732 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0620 22:59:56.036149 (+ 417us) server_negotiation.cc:299] Negotiation successful
0620 22:59:56.036540 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":685,"thread_start_us":558,"threads_started":1}
I20260620 22:59:56.037382 12229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:56.028286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.3:51343)
0620 22:59:56.028485 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:56.028506 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:56.028627 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:56.028928 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:56.028933 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:56.029327 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:56.029525 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:56.029534 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:56.030993 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:56.030997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:56.031862 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:56.031872 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:56.032763 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:56.032767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:56.032882 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:56.033383 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:56.033399 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:56.037155 (+ 3756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":60}
W20260620 22:59:56.037987 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 22:59:56.028423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:51343 (local address 127.2.237.62:37983)
0620 22:59:56.028558 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:56.028562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:56.028577 (+ 15us) server_negotiation.cc:407] Connection header received
0620 22:59:56.028742 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:56.028745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:56.028797 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:56.028871 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:56.029667 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:56.030851 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:56.031999 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:56.032625 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:56.032742 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:56.037750 (+ 5008us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.3:51343: BlockingRecv error: recv got EOF from 127.2.237.3:51343 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Jun 20 22:59:56 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 22:59:56 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 22:59:56.045773 12229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:56.038099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.3:58631)
0620 22:59:56.038299 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:56.038312 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:56.038427 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:56.038839 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:56.038844 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:56.039193 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:56.039364 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:56.039374 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:56.040573 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:56.040580 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:56.041568 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:56.041578 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:56.042225 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:56.042231 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:56.042423 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:56.043024 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:56.043040 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:56.045567 (+ 2527us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":92}
W20260620 22:59:56.045981 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 22:59:56.038284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:58631 (local address 127.2.237.62:37983)
0620 22:59:56.038527 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:56.038532 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:56.038545 (+ 13us) server_negotiation.cc:407] Connection header received
0620 22:59:56.038624 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:56.038629 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:56.038708 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:56.038785 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:56.039497 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:56.040449 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:56.041699 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:56.042076 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:56.042216 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:56.045741 (+ 3525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.3:58631: BlockingRecv error: recv got EOF from 127.2.237.3:58631 (error 108)
Metrics: {"server-negotiator.queue_time_us":138}
W20260620 22:59:56.046119 12227 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 20 22:59:56 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 22:59:56 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 22:59:56.163506 12232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:56.154322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.1:46871)
0620 22:59:56.154856 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:56.154873 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:56.155041 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:56.155439 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:56.155443 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:56.155949 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:56.156131 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:56.156140 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:56.157148 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:56.157155 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:56.158428 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:56.158440 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:56.159239 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:56.159245 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:56.159419 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:56.160125 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:56.160143 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:56.163285 (+ 3142us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":287,"threads_started":1}
W20260620 22:59:56.163812 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 22:59:56.154476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:46871 (local address 127.2.237.62:37983)
0620 22:59:56.154722 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:56.154729 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:56.155013 (+ 284us) server_negotiation.cc:407] Connection header received
0620 22:59:56.155193 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:56.155201 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:56.155278 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:56.155409 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:56.156278 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:56.157011 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:56.158611 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:56.159058 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:56.159207 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:56.163538 (+ 4331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.1:46871: BlockingRecv error: recv got EOF from 127.2.237.1:46871 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Jun 20 22:59:56 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 22:59:56 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 22:59:56.612973 12233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:56.605251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.2:41631)
0620 22:59:56.605789 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:56.605805 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:56.605915 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:56.606217 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:56.606220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:56.606700 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:56.606961 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:56.606972 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:56.607883 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:56.607886 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:56.608832 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:56.608841 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:56.609522 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:56.609526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:56.609656 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:56.610195 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:56.610211 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:56.612755 (+ 2544us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":297,"threads_started":1}
W20260620 22:59:56.613237 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 22:59:56.605299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:41631 (local address 127.2.237.62:37983)
0620 22:59:56.605491 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:56.605495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:56.605876 (+ 381us) server_negotiation.cc:407] Connection header received
0620 22:59:56.606005 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:56.606008 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:56.606070 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:56.606205 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:56.607083 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:56.607767 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:56.608978 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:56.609372 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:56.609476 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:56.612955 (+ 3479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.2:41631: BlockingRecv error: recv got EOF from 127.2.237.2:41631 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Jun 20 22:59:57 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 22:59:57 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 22:59:57.055619 12234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:57.046807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.3:60109)
0620 22:59:57.047354 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:57.047381 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:57.047518 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:57.047867 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:57.047872 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:57.048505 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:57.048770 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:57.048783 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:57.049942 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:57.049946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:57.050961 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:57.050972 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:57.051731 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:57.051736 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:57.051849 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:57.052481 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:57.052498 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:57.055362 (+ 2864us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":289,"threads_started":1}
W20260620 22:59:57.055998 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 22:59:57.046972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:60109 (local address 127.2.237.62:37983)
0620 22:59:57.047214 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:57.047220 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:57.047478 (+ 258us) server_negotiation.cc:407] Connection header received
0620 22:59:57.047632 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:57.047638 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:57.047725 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:57.047801 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:57.048942 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:57.049817 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:57.051076 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:57.051568 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:57.051662 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:57.055778 (+ 4116us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.3:60109: BlockingRecv error: recv got EOF from 127.2.237.3:60109 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Jun 20 22:59:57 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 22:59:57 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 22:59:57.172602 12235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:57.164367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.1:42641)
0620 22:59:57.164850 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:57.164867 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:57.164980 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:57.165282 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:57.165285 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:57.165661 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:57.165795 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:57.165811 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:57.167196 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:57.167200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:57.168155 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:57.168166 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:57.168828 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:57.168832 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:57.168936 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:57.169472 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:57.169490 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:57.172233 (+ 2743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":273,"threads_started":1}
W20260620 22:59:57.172680 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 22:59:57.164462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:42641 (local address 127.2.237.62:37983)
0620 22:59:57.164741 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:57.164748 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:57.164950 (+ 202us) server_negotiation.cc:407] Connection header received
0620 22:59:57.165079 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:57.165083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:57.165160 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:57.165242 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:57.165943 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:57.167069 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:57.168303 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:57.168697 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:57.168804 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:57.172422 (+ 3618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.1:42641: BlockingRecv error: recv got EOF from 127.2.237.1:42641 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Jun 20 22:59:57 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 22:59:57 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 22:59:57.621577 12236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:57.613901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.2:53489)
0620 22:59:57.614548 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:57.614563 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:57.614699 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:57.614993 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:57.614996 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:57.615369 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:57.615503 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:57.615510 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:57.616478 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:57.616481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:57.617363 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:57.617372 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:57.618027 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:57.618031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:57.618138 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:57.618707 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:57.618723 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:57.621350 (+ 2627us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":351,"threads_started":1}
W20260620 22:59:57.621884 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 22:59:57.614025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:53489 (local address 127.2.237.62:37983)
0620 22:59:57.614200 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:57.614205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:57.614684 (+ 479us) server_negotiation.cc:407] Connection header received
0620 22:59:57.614799 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:57.614803 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:57.614864 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:57.614952 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:57.615630 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:57.616359 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:57.617495 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:57.617895 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:57.617988 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:57.621602 (+ 3614us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.2:53489: BlockingRecv error: recv got EOF from 127.2.237.2:53489 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Jun 20 22:59:58 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 22:59:58 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 22:59:58.065078 12237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:58.056440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.3:53403)
0620 22:59:58.056975 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:58.056996 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:58.057109 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:58.057396 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:58.057400 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:58.057792 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:58.057934 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:58.057942 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:58.058993 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:58.058997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:58.059898 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:58.059908 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:58.060550 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:58.060554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:58.060657 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:58.061326 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:58.061343 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:58.064684 (+ 3341us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":307,"threads_started":1}
W20260620 22:59:58.065164 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 22:59:58.056610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:53403 (local address 127.2.237.62:37983)
0620 22:59:58.056801 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:58.056806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:58.057070 (+ 264us) server_negotiation.cc:407] Connection header received
0620 22:59:58.057210 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:58.057214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:58.057273 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:58.057352 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:58.058064 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:58.058874 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:58.060015 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:58.060425 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:58.060514 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:58.064885 (+ 4371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.3:53403: BlockingRecv error: recv got EOF from 127.2.237.3:53403 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Jun 20 22:59:58 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 22:59:58 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 22:59:58.181303 12238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:58.173513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.1:41305)
0620 22:59:58.174028 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:58.174044 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:58.174152 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:58.174439 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:58.174443 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:58.174829 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:58.175010 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:58.175018 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:58.175975 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:58.175979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:58.176877 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:58.176888 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:58.177528 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:58.177533 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:58.177670 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:58.178239 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:58.178257 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:58.181043 (+ 2786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":287,"threads_started":1}
W20260620 22:59:58.181490 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 22:59:58.173682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:41305 (local address 127.2.237.62:37983)
0620 22:59:58.173864 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:58.173869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:58.174119 (+ 250us) server_negotiation.cc:407] Connection header received
0620 22:59:58.174250 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:58.174254 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:58.174311 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:58.174391 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:58.175130 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:58.175855 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:58.176995 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:58.177365 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:58.177452 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:58.181231 (+ 3779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.1:41305: BlockingRecv error: recv got EOF from 127.2.237.1:41305 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Jun 20 22:59:58 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 22:59:58 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 22:59:58.631331 12239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:58.622499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.2:58065)
0620 22:59:58.622974 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:58.622990 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:58.623108 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:58.623391 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:58.623394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:58.623710 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:58.623877 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:58.623885 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:58.624877 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:58.624881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:58.625903 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:58.625914 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:58.627427 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:58.627433 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:58.627625 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:58.628306 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:58.628322 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:58.631089 (+ 2767us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":287,"threads_started":1}
W20260620 22:59:58.631548 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 22:59:58.622563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:58065 (local address 127.2.237.62:37983)
0620 22:59:58.622798 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:58.622808 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:58.623067 (+ 259us) server_negotiation.cc:407] Connection header received
0620 22:59:58.623202 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:58.623206 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:58.623266 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:58.623347 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:58.624003 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:58.624766 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:58.626050 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:58.626680 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:58.626797 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:58.631292 (+ 4495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.2:58065: BlockingRecv error: recv got EOF from 127.2.237.2:58065 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Jun 20 22:59:59 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 22:59:59 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 22:59:59.074378 12240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:59.066222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.3:35105)
0620 22:59:59.066733 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:59.066753 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:59.066907 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:59.067263 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:59.067268 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:59.067625 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:59.067779 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:59.067788 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:59.068802 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:59.068809 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:59.069701 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:59.069711 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:59.070461 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:59.070467 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:59.070677 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:59.071280 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:59.071297 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:59.074010 (+ 2713us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":264,"threads_started":1}
W20260620 22:59:59.074406 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 22:59:59.066314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:35105 (local address 127.2.237.62:37983)
0620 22:59:59.066512 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:59.066520 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:59.066840 (+ 320us) server_negotiation.cc:407] Connection header received
0620 22:59:59.067030 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:59.067035 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:59.067126 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:59.067230 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:59.067926 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:59.068662 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:59.069839 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:59.070337 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:59.070455 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:59.074197 (+ 3742us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.3:35105: BlockingRecv error: recv got EOF from 127.2.237.3:35105 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 20 22:59:59 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 22:59:59 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 22:59:59.190796 12241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:59.182264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.1:41247)
0620 22:59:59.182778 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:59.182794 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:59.182951 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:59.183277 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:59.183282 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:59.183671 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:59.183814 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:59.183823 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:59.184825 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:59.184833 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:59.185784 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:59.185795 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:59.186496 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:59.186503 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:59.186712 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:59.187275 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:59.187291 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:59.190434 (+ 3143us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":307,"threads_started":1}
W20260620 22:59:59.190990 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 22:59:59.182336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:41247 (local address 127.2.237.62:37983)
0620 22:59:59.182626 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:59.182633 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:59.182883 (+ 250us) server_negotiation.cc:407] Connection header received
0620 22:59:59.183070 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:59.183075 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:59.183146 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:59.183248 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:59.183958 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:59.184698 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:59.185925 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:59.186345 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:59.186475 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:59.190704 (+ 4229us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.1:41247: BlockingRecv error: recv got EOF from 127.2.237.1:41247 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Jun 20 22:59:59 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 22:59:59 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 22:59:59.640412 12242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 22:59:59.632309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.2:43261)
0620 22:59:59.632743 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0620 22:59:59.632759 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 22:59:59.632869 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 22:59:59.633187 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 22:59:59.633190 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 22:59:59.633538 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0620 22:59:59.633678 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:59.633685 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:59.634673 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:59.634677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:59.635635 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 22:59:59.635647 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 22:59:59.636584 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 22:59:59.636588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 22:59:59.636721 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:59.637338 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 22:59:59.637354 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 22:59:59.640167 (+ 2813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":242,"threads_started":1}
W20260620 22:59:59.640632 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 22:59:59.632370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:43261 (local address 127.2.237.62:37983)
0620 22:59:59.632625 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0620 22:59:59.632631 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 22:59:59.632834 (+ 203us) server_negotiation.cc:407] Connection header received
0620 22:59:59.632975 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 22:59:59.632978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 22:59:59.633050 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 22:59:59.633142 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0620 22:59:59.633794 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:59.634498 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:59.635775 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 22:59:59.636398 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 22:59:59.636548 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 22:59:59.640416 (+ 3868us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.2:43261: BlockingRecv error: recv got EOF from 127.2.237.2:43261 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
Jun 20 23:00:00 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:00 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:00.083886 12243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:00.075444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.3:52365)
0620 23:00:00.075981 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:00.076014 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:00.076128 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:00.076532 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:00.076537 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:00.077047 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:00.077267 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:00.077280 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:00.078366 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:00.078369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:00.079370 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:00.079381 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:00.080034 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:00.080038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:00.080161 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:00.080743 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:00.080760 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:00.083646 (+ 2886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":288,"threads_started":1}
W20260620 23:00:00.084177 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:00.075656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:52365 (local address 127.2.237.62:37983)
0620 23:00:00.075875 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:00.075882 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:00.076105 (+ 223us) server_negotiation.cc:407] Connection header received
0620 23:00:00.076277 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:00.076282 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:00.076364 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:00.076464 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:00.077393 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:00.078243 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:00.079499 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:00.079911 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:00.080002 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:00.083919 (+ 3917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.3:52365: BlockingRecv error: recv got EOF from 127.2.237.3:52365 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Jun 20 23:00:00 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:00 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:00.216318 12244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:00.191709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.1:50213)
0620 23:00:00.192245 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:00.192261 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:00.192450 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:00.192803 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:00.192808 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:00.193226 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:00.193388 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:00.193396 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:00.194527 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:00.194531 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:00.195729 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:00.195741 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:00.196451 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:00.196455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:00.196580 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:00.213049 (+ 16469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:00.213071 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:00.216080 (+ 3009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":267,"threads_started":1}
W20260620 23:00:00.216472 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:00.191778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:50213 (local address 127.2.237.62:37983)
0620 23:00:00.192059 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:00.192066 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:00.192350 (+ 284us) server_negotiation.cc:407] Connection header received
0620 23:00:00.192586 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:00.192592 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:00.192661 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:00.192767 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:00.193544 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:00.194375 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:00.195841 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:00.196336 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:00.196435 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:00.216282 (+ 19847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.1:50213: BlockingRecv error: recv got EOF from 127.2.237.1:50213 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Jun 20 23:00:00 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:00 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:00.649956 12245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:00.641424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.2:40371)
0620 23:00:00.641957 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:00.641972 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:00.642122 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:00.642449 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:00.642455 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:00.642940 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:00.643116 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:00.643125 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:00.644177 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:00.644185 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:00.645105 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:00.645116 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:00.645834 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:00.645841 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:00.646008 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:00.646597 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:00.646614 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:00.649606 (+ 2992us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":257,"threads_started":1}
W20260620 23:00:00.650082 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:00.641608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:40371 (local address 127.2.237.62:37983)
0620 23:00:00.641799 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:00.641806 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:00.642059 (+ 253us) server_negotiation.cc:407] Connection header received
0620 23:00:00.642243 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:00.642248 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:00.642315 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:00.642415 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:00.643258 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:00.644033 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:00.645247 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:00.645680 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:00.645805 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:00.649825 (+ 4020us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.2:40371: BlockingRecv error: recv got EOF from 127.2.237.2:40371 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Jun 20 23:00:01 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:01 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:01.093542 12246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:01.084881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.3:38885)
0620 23:00:01.085372 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:01.085401 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:01.085513 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:01.085900 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:01.085903 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:01.086285 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:01.086475 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:01.086483 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:01.087595 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:01.087599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:01.088578 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:01.088614 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:01.089254 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:01.089259 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:01.089372 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:01.090163 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:01.090189 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:01.093300 (+ 3111us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":244,"threads_started":1}
W20260620 23:00:01.093667 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:01.084962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:38885 (local address 127.2.237.62:37983)
0620 23:00:01.085209 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:01.085216 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:01.085485 (+ 269us) server_negotiation.cc:407] Connection header received
0620 23:00:01.085673 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:01.085677 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:01.085761 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:01.085862 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:01.086658 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:01.087444 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:01.088738 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:01.089133 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:01.089220 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:01.093461 (+ 4241us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.3:38885: BlockingRecv error: recv got EOF from 127.2.237.3:38885 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Jun 20 23:00:01 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:01 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:01.225700 12247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:01.217354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.1:39401)
0620 23:00:01.217889 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:01.217907 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:01.218071 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:01.218455 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:01.218460 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:01.218874 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:01.219032 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:01.219040 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:01.220077 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:01.220085 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:01.220994 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:01.221004 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:01.221742 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:01.221748 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:01.221927 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:01.222694 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:01.222713 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:01.225487 (+ 2774us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":306,"threads_started":1}
W20260620 23:00:01.225978 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:01.217478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:39401 (local address 127.2.237.62:37983)
0620 23:00:01.217740 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:01.217748 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:01.217999 (+ 251us) server_negotiation.cc:407] Connection header received
0620 23:00:01.218200 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:01.218205 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:01.218276 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:01.218382 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:01.219182 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:01.219938 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:01.221136 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:01.221588 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:01.221730 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:01.225700 (+ 3970us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.1:39401: BlockingRecv error: recv got EOF from 127.2.237.1:39401 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Jun 20 23:00:01 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:01 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:01.659322 12248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:01.651120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.2:45493)
0620 23:00:01.651725 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:01.651741 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:01.651858 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:01.652218 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:01.652221 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:01.652602 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:01.652749 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:01.652756 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:01.653688 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:01.653691 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:01.654700 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:01.654711 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:01.655444 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:01.655451 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:01.655631 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:01.656160 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:01.656176 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:01.658998 (+ 2822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":376,"threads_started":1}
W20260620 23:00:01.659466 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:01.651165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:45493 (local address 127.2.237.62:37983)
0620 23:00:01.651295 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:01.651299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:01.651811 (+ 512us) server_negotiation.cc:407] Connection header received
0620 23:00:01.651954 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:01.651971 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:01.652087 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:01.652178 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:01.652878 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:01.653570 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:01.654818 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:01.655267 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:01.655400 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:01.659226 (+ 3826us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.2:45493: BlockingRecv error: recv got EOF from 127.2.237.2:45493 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Jun 20 23:00:02 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:02 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:02.103375 12249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:02.094662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.3:43463)
0620 23:00:02.095143 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:02.095163 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:02.095278 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:02.095611 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:02.095615 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:02.096040 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:02.096194 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:02.096203 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:02.097463 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:02.097467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:02.098368 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:02.098379 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:02.099269 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:02.099275 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:02.099483 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:02.100199 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:02.100217 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:02.103153 (+ 2936us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":272,"threads_started":1}
W20260620 23:00:02.103623 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:02.094725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:43463 (local address 127.2.237.62:37983)
0620 23:00:02.094988 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:02.094995 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:02.095244 (+ 249us) server_negotiation.cc:407] Connection header received
0620 23:00:02.095380 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:02.095386 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:02.095462 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:02.095556 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:02.096320 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:02.097279 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:02.098535 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:02.099074 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:02.099229 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:02.103356 (+ 4127us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.3:43463: BlockingRecv error: recv got EOF from 127.2.237.3:43463 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Jun 20 23:00:02 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:02 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:02.235282 12250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:02.226799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.1:42477)
0620 23:00:02.227476 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:02.227492 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:02.227606 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:02.227894 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:02.227898 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:02.228317 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:02.228475 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:02.228483 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:02.229445 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:02.229448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:02.230358 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:02.230367 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:02.231020 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:02.231024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:02.231146 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:02.231793 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:02.231811 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:02.234907 (+ 3096us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":404,"threads_started":1}
W20260620 23:00:02.235458 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:02.226852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:42477 (local address 127.2.237.62:37983)
0620 23:00:02.227146 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:02.227155 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:02.227577 (+ 422us) server_negotiation.cc:407] Connection header received
0620 23:00:02.227704 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:02.227707 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:02.227770 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:02.227854 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:02.228610 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:02.229321 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:02.230486 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:02.230893 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:02.230987 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:02.235152 (+ 4165us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.1:42477: BlockingRecv error: recv got EOF from 127.2.237.1:42477 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Jun 20 23:00:02 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:02 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:02.669083 12251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:02.660503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.2:35583)
0620 23:00:02.661140 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:02.661156 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:02.661316 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:02.661686 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:02.661690 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:02.662127 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:02.662286 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:02.662295 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:02.663526 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:02.663532 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:02.664443 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:02.664454 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:02.665169 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:02.665175 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:02.665354 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:02.665913 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:02.665930 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:02.668672 (+ 2742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":301,"threads_started":1}
W20260620 23:00:02.669119 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:02.660612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:35583 (local address 127.2.237.62:37983)
0620 23:00:02.660894 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:02.660903 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:02.661251 (+ 348us) server_negotiation.cc:407] Connection header received
0620 23:00:02.661476 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:02.661480 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:02.661550 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:02.661661 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:02.662434 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:02.663381 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:02.664588 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:02.665012 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:02.665156 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:02.668869 (+ 3713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.2:35583: BlockingRecv error: recv got EOF from 127.2.237.2:35583 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Jun 20 23:00:03 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:03 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:03.113564 12252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:03.104346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.3:46045)
0620 23:00:03.104909 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:03.104928 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:03.105077 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:03.105416 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:03.105419 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:03.105816 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:03.105975 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:03.105984 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:03.107387 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:03.107391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:03.108472 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:03.108484 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:03.109218 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:03.109222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:03.109327 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:03.109939 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:03.109955 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:03.113261 (+ 3306us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":228,"threads_started":1}
W20260620 23:00:03.113706 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:03.104399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:46045 (local address 127.2.237.62:37983)
0620 23:00:03.104618 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:03.104628 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:03.105008 (+ 380us) server_negotiation.cc:407] Connection header received
0620 23:00:03.105216 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:03.105220 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:03.105284 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:03.105398 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:03.106116 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:03.107251 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:03.108601 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:03.109093 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:03.109195 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:03.113510 (+ 4315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.3:46045: BlockingRecv error: recv got EOF from 127.2.237.3:46045 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Jun 20 23:00:03 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:03 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:03.245205 12253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:03.236380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.1:58795)
0620 23:00:03.236946 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:03.236961 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:03.237123 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:03.237467 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:03.237472 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:03.237884 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:03.238032 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:03.238041 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:03.239192 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:03.239197 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:03.240348 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:03.240363 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:03.241199 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:03.241205 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:03.241371 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:03.242017 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:03.242034 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:03.244974 (+ 2940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":296,"threads_started":1}
W20260620 23:00:03.245432 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:03.236582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:58795 (local address 127.2.237.62:37983)
0620 23:00:03.236817 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:03.236827 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:03.237060 (+ 233us) server_negotiation.cc:407] Connection header received
0620 23:00:03.237249 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:03.237255 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:03.237327 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:03.237437 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:03.238179 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:03.239036 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:03.240499 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:03.241017 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:03.241166 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:03.245169 (+ 4003us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.1:58795: BlockingRecv error: recv got EOF from 127.2.237.1:58795 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Jun 20 23:00:03 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:03 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:03.678635 12254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:03.670109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.2:52425)
0620 23:00:03.670763 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:03.670784 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:03.670916 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:03.671237 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:03.671243 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:03.671617 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:03.671766 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:03.671774 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:03.672881 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:03.672884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:03.673755 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:03.673765 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:03.674395 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:03.674399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:03.674512 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:03.675210 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:03.675227 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:03.678285 (+ 3058us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":335,"threads_started":1}
W20260620 23:00:03.678728 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:03.670225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:52425 (local address 127.2.237.62:37983)
0620 23:00:03.670465 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:03.670473 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:03.670881 (+ 408us) server_negotiation.cc:407] Connection header received
0620 23:00:03.671042 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:03.671049 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:03.671117 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:03.671195 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:03.671884 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:03.672763 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:03.673870 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:03.674253 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:03.674342 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:03.678463 (+ 4121us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.2:52425: BlockingRecv error: recv got EOF from 127.2.237.2:52425 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Jun 20 23:00:04 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:04 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:04.122516 12255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:04.114614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.3:46033)
0620 23:00:04.115119 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:04.115139 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:04.115292 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:04.115635 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:04.115641 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:04.116080 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:04.116274 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:04.116302 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:04.117268 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:04.117272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:04.118182 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:04.118191 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:04.118956 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:04.118963 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:04.119091 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:04.119643 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:04.119660 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:04.122283 (+ 2623us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":244,"threads_started":1}
W20260620 23:00:04.122784 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:04.114681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:46033 (local address 127.2.237.62:37983)
0620 23:00:04.114885 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:04.114891 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:04.115222 (+ 331us) server_negotiation.cc:407] Connection header received
0620 23:00:04.115412 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:04.115415 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:04.115476 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:04.115559 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:04.116408 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:04.117138 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:04.118298 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:04.118759 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:04.118883 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:04.122472 (+ 3589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.3:46033: BlockingRecv error: recv got EOF from 127.2.237.3:46033 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Jun 20 23:00:04 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:04 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:04.254894 12256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:04.246194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.1:48451)
0620 23:00:04.246844 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:04.246863 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:04.247023 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:04.247418 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:04.247421 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:04.247865 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:04.248117 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:04.248131 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:04.249275 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:04.249279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:04.250219 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:04.250229 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:04.250961 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:04.250966 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:04.251093 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:04.251736 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:04.251753 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:04.254560 (+ 2807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":388,"threads_started":1}
W20260620 23:00:04.255259 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:04.246276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:48451 (local address 127.2.237.62:37983)
0620 23:00:04.246512 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:04.246519 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:04.246971 (+ 452us) server_negotiation.cc:407] Connection header received
0620 23:00:04.247178 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:04.247182 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:04.247244 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:04.247366 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:04.248271 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:04.249157 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:04.250339 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:04.250810 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:04.250905 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:04.254844 (+ 3939us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.1:48451: BlockingRecv error: recv got EOF from 127.2.237.1:48451 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Jun 20 23:00:04 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:04 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:04.688336 12257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:04.679722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.2:38291)
0620 23:00:04.680319 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:04.680335 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:04.680486 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:04.680792 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:04.680795 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:04.681163 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:04.681314 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:04.681322 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:04.682352 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:04.682356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:04.683385 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:04.683396 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:04.684088 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:04.684092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:04.684217 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:04.684844 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:04.684871 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:04.687967 (+ 3096us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":338,"threads_started":1}
W20260620 23:00:04.688406 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:04.679866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:38291 (local address 127.2.237.62:37983)
0620 23:00:04.680039 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:04.680043 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:04.680445 (+ 402us) server_negotiation.cc:407] Connection header received
0620 23:00:04.680592 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:04.680596 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:04.680655 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:04.680748 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:04.681478 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:04.682209 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:04.683517 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:04.683947 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:04.684037 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:04.688160 (+ 4123us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.2:38291: BlockingRecv error: recv got EOF from 127.2.237.2:38291 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Jun 20 23:00:05 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:05 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:05.132138 12258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:05.123576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.3:50813)
0620 23:00:05.124098 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:05.124118 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:05.124280 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:05.124694 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:05.124700 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:05.125260 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:05.125423 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:05.125433 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:05.126442 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:05.126449 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:05.127416 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:05.127427 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:05.128148 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:05.128154 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:05.128335 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:05.128948 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:05.128965 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:05.131784 (+ 2819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":271,"threads_started":1}
W20260620 23:00:05.132210 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:05.123640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:50813 (local address 127.2.237.62:37983)
0620 23:00:05.123917 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:05.123923 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:05.124214 (+ 291us) server_negotiation.cc:407] Connection header received
0620 23:00:05.124405 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:05.124411 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:05.124518 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:05.124622 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:05.125573 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:05.126309 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:05.127563 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:05.127977 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:05.128136 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:05.131976 (+ 3840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.3:50813: BlockingRecv error: recv got EOF from 127.2.237.3:50813 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Jun 20 23:00:05 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:05 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:05.269462 12259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:05.255954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.1:53695)
0620 23:00:05.256505 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:05.256523 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:05.256681 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:05.257040 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:05.257045 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:05.257426 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:05.257577 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:05.257586 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:05.258833 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:05.258841 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:05.259778 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:05.259790 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:05.260565 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:05.260571 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:05.260744 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:05.265838 (+ 5094us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:05.265860 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:05.269156 (+ 3296us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":327,"threads_started":1}
W20260620 23:00:05.269698 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:05.256183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:53695 (local address 127.2.237.62:37983)
0620 23:00:05.256405 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:05.256412 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:05.256638 (+ 226us) server_negotiation.cc:407] Connection header received
0620 23:00:05.256807 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:05.256811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:05.256876 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:05.256976 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:05.257733 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:05.258704 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:05.259929 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:05.260397 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:05.260537 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:05.269399 (+ 8862us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.1:53695: BlockingRecv error: recv got EOF from 127.2.237.1:53695 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Jun 20 23:00:05 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:05 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:05.697854 12260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:05.689297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.2:48205)
0620 23:00:05.689774 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:05.689791 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:05.689918 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:05.690219 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:05.690222 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:05.690612 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:05.690749 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:05.690757 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:05.691753 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:05.691757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:05.692668 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:05.692678 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:05.693387 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:05.693394 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:05.693529 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:05.694222 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:05.694239 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:05.697516 (+ 3277us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":279,"threads_started":1}
W20260620 23:00:05.697948 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:05.689486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:48205 (local address 127.2.237.62:37983)
0620 23:00:05.689743 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:05.689749 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:05.689887 (+ 138us) server_negotiation.cc:407] Connection header received
0620 23:00:05.690016 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:05.690020 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:05.690098 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:05.690180 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:05.690888 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:05.691619 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:05.692787 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:05.693212 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:05.693302 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:05.697709 (+ 4407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.2:48205: BlockingRecv error: recv got EOF from 127.2.237.2:48205 (error 108)
Metrics: {"server-negotiator.queue_time_us":116}
Jun 20 23:00:06 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:06 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:06.142112 12261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:06.133213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.3:49181)
0620 23:00:06.133818 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:06.133854 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:06.134028 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:06.134379 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:06.134384 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:06.134828 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:06.134970 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:06.134978 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:06.136021 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:06.136028 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:06.137087 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:06.137097 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:06.137937 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:06.137944 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:06.138126 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:06.138809 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:06.138826 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:06.141882 (+ 3056us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":340,"threads_started":1}
W20260620 23:00:06.142316 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:06.133489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:49181 (local address 127.2.237.62:37983)
0620 23:00:06.133788 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:06.133800 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:06.133953 (+ 153us) server_negotiation.cc:407] Connection header received
0620 23:00:06.134164 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:06.134176 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:06.134236 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:06.134349 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:06.135125 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:06.135865 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:06.137237 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:06.137776 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:06.137912 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:06.142088 (+ 4176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.3:49181: BlockingRecv error: recv got EOF from 127.2.237.3:49181 (error 108)
Metrics: {"server-negotiator.queue_time_us":159}
Jun 20 23:00:06 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:06 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:06.278934 12262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:06.270401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.1:38521)
0620 23:00:06.271154 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:06.271169 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:06.271327 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:06.271871 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:06.271879 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:06.272353 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:06.272520 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:06.272529 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:06.273523 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:06.273526 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:06.274401 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:06.274410 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:06.275150 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:06.275154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:06.275270 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:06.275924 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:06.275943 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:06.278694 (+ 2751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":513,"threads_started":1}
W20260620 23:00:06.279397 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:06.270638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:38521 (local address 127.2.237.62:37983)
0620 23:00:06.270891 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:06.270898 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:06.271261 (+ 363us) server_negotiation.cc:407] Connection header received
0620 23:00:06.271469 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:06.271474 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:06.271680 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:06.271813 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:06.272667 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:06.273403 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:06.274535 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:06.275022 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:06.275127 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:06.278926 (+ 3799us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.1:38521: BlockingRecv error: recv got EOF from 127.2.237.1:38521 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Jun 20 23:00:06 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:06 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:06.709868 12263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:06.698831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.2:56953)
0620 23:00:06.699351 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:06.699371 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:06.699516 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:06.699845 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:06.699849 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:06.700272 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:06.700451 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:06.700459 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:06.702004 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:06.702011 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:06.703774 (+ 1763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:06.703792 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:06.704895 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:06.704899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:06.705069 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:06.705770 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:06.705794 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:06.709503 (+ 3709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":275,"threads_started":1}
W20260620 23:00:06.709923 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:06.698880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:56953 (local address 127.2.237.62:37983)
0620 23:00:06.699065 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:06.699071 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:06.699443 (+ 372us) server_negotiation.cc:407] Connection header received
0620 23:00:06.699639 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:06.699642 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:06.699704 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:06.699783 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:06.700577 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:06.701842 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:06.703939 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:06.704681 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:06.704821 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:06.709694 (+ 4873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.2:56953: BlockingRecv error: recv got EOF from 127.2.237.2:56953 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Jun 20 23:00:07 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:07 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:07.151644 12264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:07.143205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.3:39963)
0620 23:00:07.143766 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:07.143793 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:07.143915 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:07.144218 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:07.144221 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:07.144644 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:07.144817 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:07.144825 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:07.145834 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:07.145842 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:07.146773 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:07.146784 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:07.147504 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:07.147510 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:07.147695 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:07.148423 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:07.148440 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:07.151396 (+ 2956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":320,"threads_started":1}
W20260620 23:00:07.151748 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:07.143271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:39963 (local address 127.2.237.62:37983)
0620 23:00:07.143461 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:07.143470 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:07.143893 (+ 423us) server_negotiation.cc:407] Connection header received
0620 23:00:07.144011 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:07.144028 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:07.144090 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:07.144169 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:07.144981 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:07.145699 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:07.146920 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:07.147331 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:07.147461 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:07.151549 (+ 4088us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.3:39963: BlockingRecv error: recv got EOF from 127.2.237.3:39963 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Jun 20 23:00:07 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:07 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:07.288543 12265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:07.279817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.1:50219)
0620 23:00:07.280359 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:07.280375 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:07.280488 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:07.280813 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:07.280816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:07.281208 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:07.281364 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:07.281372 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:07.282346 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:07.282350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:07.283279 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:07.283290 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:07.284009 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:07.284014 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:07.284138 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:07.284937 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:07.284954 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:07.288142 (+ 3188us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":308,"threads_started":1}
W20260620 23:00:07.288614 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:07.279953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:50219 (local address 127.2.237.62:37983)
0620 23:00:07.280135 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:07.280141 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:07.280453 (+ 312us) server_negotiation.cc:407] Connection header received
0620 23:00:07.280606 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:07.280609 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:07.280670 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:07.280767 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:07.281491 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:07.282223 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:07.283426 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:07.283882 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:07.283985 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:07.288352 (+ 4367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.1:50219: BlockingRecv error: recv got EOF from 127.2.237.1:50219 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Jun 20 23:00:07 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:07 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:07.720129 12266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:07.711600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.2:50833)
0620 23:00:07.712161 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:07.712177 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:07.712340 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:07.712696 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:07.712701 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:07.713102 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:07.713252 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:07.713260 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:07.714284 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:07.714292 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:07.715540 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:07.715555 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:07.716338 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:07.716344 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:07.716471 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:07.717130 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:07.717148 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:07.719881 (+ 2733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":313,"threads_started":1}
W20260620 23:00:07.720266 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:07.711701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:50833 (local address 127.2.237.62:37983)
0620 23:00:07.711993 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:07.712002 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:07.712273 (+ 271us) server_negotiation.cc:407] Connection header received
0620 23:00:07.712491 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:07.712495 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:07.712556 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:07.712662 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:07.713408 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:07.714142 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:07.715665 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:07.716207 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:07.716304 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:07.720063 (+ 3759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.2:50833: BlockingRecv error: recv got EOF from 127.2.237.2:50833 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Jun 20 23:00:08 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:08 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:08.161648 12267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:08.152834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.3:50463)
0620 23:00:08.153410 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:08.153431 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:08.153549 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:08.153918 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:08.153922 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:08.154325 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:08.154544 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:08.154554 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:08.155674 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:08.155679 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:08.156732 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:08.156745 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:08.157466 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:08.157471 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:08.157620 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:08.158295 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:08.158312 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:08.161394 (+ 3082us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":308,"threads_started":1}
W20260620 23:00:08.161903 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:08.152860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:50463 (local address 127.2.237.62:37983)
0620 23:00:08.153045 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:08.153051 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:08.153505 (+ 454us) server_negotiation.cc:407] Connection header received
0620 23:00:08.153654 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:08.153680 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:08.153761 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:08.153856 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:08.154694 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:08.155492 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:08.156874 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:08.157324 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:08.157422 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:08.161628 (+ 4206us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.3:50463: BlockingRecv error: recv got EOF from 127.2.237.3:50463 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Jun 20 23:00:08 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:08 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:08.298048 12268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:08.289654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.1:55041)
0620 23:00:08.290316 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:08.290333 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:08.290445 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:08.290761 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:08.290765 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:08.291133 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:08.291284 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:08.291293 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:08.292347 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:08.292352 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:08.293298 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:08.293343 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:08.294107 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:08.294112 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:08.294235 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:08.294848 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:08.294870 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:08.297800 (+ 2930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":352,"threads_started":1}
W20260620 23:00:08.298224 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:08.289729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:55041 (local address 127.2.237.62:37983)
0620 23:00:08.289949 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:08.289956 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:08.290408 (+ 452us) server_negotiation.cc:407] Connection header received
0620 23:00:08.290542 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:08.290546 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:08.290624 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:08.290701 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:08.291449 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:08.292163 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:08.293491 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:08.293924 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:08.294033 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:08.297974 (+ 3941us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.1:55041: BlockingRecv error: recv got EOF from 127.2.237.1:55041 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Jun 20 23:00:08 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:08 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:08.729794 12269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:08.721086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.2:45239)
0620 23:00:08.721663 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:08.721680 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:08.721852 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:08.722173 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:08.722178 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:08.722656 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:08.722825 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:08.722841 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:08.723867 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:08.723874 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:08.724821 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:08.724840 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:08.725615 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:08.725622 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:08.725797 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:08.726374 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:08.726391 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:08.729436 (+ 3045us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":308,"threads_started":1}
W20260620 23:00:08.729887 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:08.721298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:45239 (local address 127.2.237.62:37983)
0620 23:00:08.721548 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:08.721554 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:08.721759 (+ 205us) server_negotiation.cc:407] Connection header received
0620 23:00:08.721976 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:08.721986 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:08.722046 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:08.722149 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:08.722974 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:08.723717 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:08.724979 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:08.725429 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:08.725569 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:08.729633 (+ 4064us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.2:45239: BlockingRecv error: recv got EOF from 127.2.237.2:45239 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Jun 20 23:00:09 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:09 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:09.171025 12270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:09.162623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.3:36393)
0620 23:00:09.163147 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:09.163173 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:09.163404 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:09.163751 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:09.163758 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:09.164174 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:09.164350 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:09.164358 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:09.165368 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:09.165376 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:09.166300 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:09.166310 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:09.167126 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:09.167132 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:09.167297 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:09.167863 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:09.167880 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:09.170665 (+ 2785us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":272,"threads_started":1}
W20260620 23:00:09.171072 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:09.162694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:36393 (local address 127.2.237.62:37983)
0620 23:00:09.162926 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:09.162932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:09.163266 (+ 334us) server_negotiation.cc:407] Connection header received
0620 23:00:09.163540 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:09.163546 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:09.163615 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:09.163717 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:09.164494 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:09.165209 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:09.166449 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:09.166946 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:09.167084 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:09.170847 (+ 3763us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.3:36393: BlockingRecv error: recv got EOF from 127.2.237.3:36393 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Jun 20 23:00:09 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:09 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:09.307475 12271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:09.299024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.1:57395)
0620 23:00:09.299607 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:09.299627 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:09.299793 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:09.300150 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:09.300154 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:09.300572 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:09.300736 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:09.300744 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:09.301809 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:09.301816 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:09.302928 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:09.302943 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:09.303620 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:09.303625 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:09.303793 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:09.304431 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:09.304448 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:09.307229 (+ 2781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":282,"threads_started":1}
W20260620 23:00:09.307878 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:09.299107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:57395 (local address 127.2.237.62:37983)
0620 23:00:09.299454 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:09.299466 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:09.299732 (+ 266us) server_negotiation.cc:407] Connection header received
0620 23:00:09.299940 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:09.299951 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:09.300024 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:09.300125 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:09.300885 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:09.301670 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:09.303066 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:09.303475 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:09.303611 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:09.307426 (+ 3815us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.1:57395: BlockingRecv error: recv got EOF from 127.2.237.1:57395 (error 108)
Metrics: {"server-negotiator.queue_time_us":156}
Jun 20 23:00:09 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:09 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:09.739516 12272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:09.730910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.2:37341)
0620 23:00:09.731486 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:09.731502 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:09.731678 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:09.732051 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:09.732057 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:09.732445 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:09.732632 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:09.732640 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:09.733676 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:09.733683 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:09.734790 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:09.734823 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:09.735724 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:09.735729 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:09.735851 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:09.736477 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:09.736494 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:09.739253 (+ 2759us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":295,"threads_started":1}
W20260620 23:00:09.739666 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:09.731146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:37341 (local address 127.2.237.62:37983)
0620 23:00:09.731395 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:09.731402 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:09.731593 (+ 191us) server_negotiation.cc:407] Connection header received
0620 23:00:09.731811 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:09.731818 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:09.731886 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:09.732010 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:09.732785 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:09.733526 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:09.735001 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:09.735590 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:09.735687 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:09.739459 (+ 3772us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.2:37341: BlockingRecv error: recv got EOF from 127.2.237.2:37341 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Jun 20 23:00:10 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:10 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.3@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:10.181773 12273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:10.172185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.3:42569)
0620 23:00:10.172749 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:10.172771 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:10.172901 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:10.173513 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:10.173517 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:10.173969 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:10.174244 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:10.174255 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:10.175727 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:10.175733 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:10.176732 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:10.176746 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:10.177426 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:10.177431 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:10.177567 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:10.178337 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:10.178357 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:10.181509 (+ 3152us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":302,"threads_started":1}
W20260620 23:00:10.182127 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:10.172332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:42569 (local address 127.2.237.62:37983)
0620 23:00:10.172513 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:10.172518 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:10.172903 (+ 385us) server_negotiation.cc:407] Connection header received
0620 23:00:10.173069 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:10.173074 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:10.173155 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:10.173304 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:10.174397 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:10.175555 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:10.176882 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:10.177279 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:10.177371 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:10.181741 (+ 4370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.3:42569: BlockingRecv error: recv got EOF from 127.2.237.3:42569 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Jun 20 23:00:10 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:10 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.1@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:10.320539 12274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:10.308805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.1:44707)
0620 23:00:10.309383 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:10.309398 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:10.309570 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:10.309935 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:10.309940 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:10.310376 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:10.310563 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:10.310632 (+ 69us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:10.311866 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:10.311871 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:10.312989 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:10.313001 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:10.313723 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:10.313729 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:10.313925 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:10.317217 (+ 3292us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:10.317245 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:10.320266 (+ 3021us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":346,"threads_started":1}
W20260620 23:00:10.320803 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:10.308939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:44707 (local address 127.2.237.62:37983)
0620 23:00:10.309187 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:10.309195 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:10.309492 (+ 297us) server_negotiation.cc:407] Connection header received
0620 23:00:10.309712 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:10.309716 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:10.309790 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:10.309903 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:10.310787 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:10.311671 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:10.313122 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:10.313570 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:10.313699 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:10.320559 (+ 6860us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.1:44707: BlockingRecv error: recv got EOF from 127.2.237.1:44707 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Jun 20 23:00:10 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
Jun 20 23:00:10 dist-test-slave-f63b krb5kdc[11229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.237.2@KRBTEST.COM for oryx/127.2.237.62@KRBTEST.COM, Server not found in Kerberos database
I20260620 23:00:10.749763 12275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:10.740663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37983 (local address 127.2.237.2:37145)
0620 23:00:10.741211 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:10.741227 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:10.741334 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:10.741673 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:10.741677 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:10.742129 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:10.742321 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:10.742329 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:10.743586 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:10.743590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:10.744607 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:10.744619 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:10.745422 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:10.745427 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:10.745552 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:10.746135 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:10.746152 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:10.749510 (+ 3358us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.237.62:37983: Server oryx/127.2.237.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":324,"threads_started":1}
W20260620 23:00:10.749962 11966 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:10.740806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:37145 (local address 127.2.237.62:37983)
0620 23:00:10.740992 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:10.740997 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:10.741308 (+ 311us) server_negotiation.cc:407] Connection header received
0620 23:00:10.741449 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:10.741452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:10.741517 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:10.741606 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:10.742469 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:10.743392 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:10.744758 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:10.745220 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:10.745322 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:10.749697 (+ 4375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.237.2:37145: BlockingRecv error: recv got EOF from 127.2.237.2:37145 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260620 23:00:11.020355 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 11816
I20260620 23:00:11.033725 12277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:11.021830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35466 (local address 127.2.237.1:45659)
0620 23:00:11.022519 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:11.022530 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:11.022595 (+ 65us) server_negotiation.cc:407] Connection header received
0620 23:00:11.022683 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:11.022693 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:11.022994 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:11.023236 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:11.024790 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:11.026252 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:11.027523 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:11.028130 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:11.028243 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:11.030694 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:00:11.030703 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:00:11.030718 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:00:11.030756 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:00:11.033160 (+ 2404us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.2.237.1@KRBTEST.COM not found in keytab (ticket kvno 2)
0620 23:00:11.033479 (+ 319us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35466: Request ticket server kudu/127.2.237.1@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":298,"threads_started":1}
W20260620 23:00:11.033876 12277 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35466: Request ticket server kudu/127.2.237.1@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260620 23:00:11.033879 2996 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 11816: Not authorized: Client connection negotiation failed: client connection to 127.2.237.1:45659: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.2.237.1@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260620 23:00:11.034133 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 11816
I20260620 23:00:11.053730 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 11968
I20260620 23:00:11.065768 12278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:11.054896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58882 (local address 127.2.237.2:44543)
0620 23:00:11.055535 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:11.055543 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:11.055574 (+ 31us) server_negotiation.cc:407] Connection header received
0620 23:00:11.055633 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:11.055640 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:11.055863 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:11.056017 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:11.057251 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:11.058366 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:11.059597 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:11.060128 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:11.060223 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:11.062693 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:00:11.062702 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:00:11.062717 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:00:11.062752 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:00:11.065198 (+ 2446us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.2.237.2@KRBTEST.COM not found in keytab (ticket kvno 2)
0620 23:00:11.065459 (+ 261us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58882: Request ticket server kudu/127.2.237.2@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":319,"threads_started":1}
W20260620 23:00:11.065876 12278 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58882: Request ticket server kudu/127.2.237.2@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260620 23:00:11.065860 2996 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 11968: Not authorized: Client connection negotiation failed: client connection to 127.2.237.2:44543: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.2.237.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260620 23:00:11.066026 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 11968
I20260620 23:00:11.082666 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 12099
I20260620 23:00:11.095003 12279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:11.083587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54478 (local address 127.2.237.3:45241)
0620 23:00:11.084255 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:11.084273 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:11.084301 (+ 28us) server_negotiation.cc:407] Connection header received
0620 23:00:11.084363 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:11.084369 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:11.084599 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:11.084760 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:11.086021 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:11.087275 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:11.088621 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:11.089160 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:11.089270 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:11.091928 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:00:11.091937 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:00:11.091952 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:00:11.091991 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:00:11.094424 (+ 2433us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.2.237.3@KRBTEST.COM not found in keytab (ticket kvno 2)
0620 23:00:11.094762 (+ 338us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54478: Request ticket server kudu/127.2.237.3@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":390,"threads_started":1}
W20260620 23:00:11.095129 12279 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54478: Request ticket server kudu/127.2.237.3@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260620 23:00:11.095116 2996 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 12099: Not authorized: Client connection negotiation failed: client connection to 127.2.237.3:45241: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.2.237.3@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260620 23:00:11.095294 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 12099
I20260620 23:00:11.111811 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 11747
I20260620 23:00:11.113091 11809 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:00:11.228572 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 11747
2026-06-20T23:00:11Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21023 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-20T23:00:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:00:11Z Disabled control of system clock
I20260620 23:00:11.308061 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:41787
--webserver_interface=127.2.237.62
--webserver_port=0
--builtin_ntp_servers=127.2.237.20:41599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:41787
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260620 23:00:11.643576 12284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:00:11.643872 12284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:00:11.644035 12284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:00:11.651439 12284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:00:11.651561 12284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:00:11.651631 12284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:00:11.651717 12284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:00:11.651782 12284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:00:11.661240 12284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:41599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:41787
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:41787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:00:11.662678 12284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:00:11.664549 12284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:00:11.676957 12289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:00:11.676883 12292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:00:11.676909 12290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:00:11.677752 12284 server_base.cc:1034] running on GCE node
I20260620 23:00:11.678562 12284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:00:11.680155 12284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:00:11.681397 12284 hybrid_clock.cc:648] HybridClock initialized: now 1781996411681347 us; error 43 us; skew 500 ppm
I20260620 23:00:11.681766 12284 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 23:00:11.684165 12284 webserver.cc:460] Webserver started at http://127.2.237.62:42625/ using document root <none> and password file <none>
I20260620 23:00:11.684937 12284 fs_manager.cc:359] Metadata directory not provided
I20260620 23:00:11.685071 12284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:00:11.685410 12284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:00:11.688414 12284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "3e93ff2139be43f48287bb6a094b44a9"
format_stamp: "Formatted at 2026-06-20 23:00:11 on dist-test-slave-f63b"
server_key: "da9c019d8dd411588c9095a4422b9262"
server_key_iv: "31325ba632ca78d175b617288ad08cc5"
server_key_version: "encryptionkey@0"
I20260620 23:00:11.689206 12284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "3e93ff2139be43f48287bb6a094b44a9"
format_stamp: "Formatted at 2026-06-20 23:00:11 on dist-test-slave-f63b"
server_key: "da9c019d8dd411588c9095a4422b9262"
server_key_iv: "31325ba632ca78d175b617288ad08cc5"
server_key_version: "encryptionkey@0"
I20260620 23:00:11.694697 12284 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260620 23:00:11.697999 12298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:00:11.699415 12284 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260620 23:00:11.699638 12284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "3e93ff2139be43f48287bb6a094b44a9"
format_stamp: "Formatted at 2026-06-20 23:00:11 on dist-test-slave-f63b"
server_key: "da9c019d8dd411588c9095a4422b9262"
server_key_iv: "31325ba632ca78d175b617288ad08cc5"
server_key_version: "encryptionkey@0"
I20260620 23:00:11.699856 12284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:00:11.727933 12284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:00:11.728859 12284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:00:11.729173 12284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:00:11.747534 12284 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:41787
I20260620 23:00:11.747565 12349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:41787 every 8 connection(s)
I20260620 23:00:11.748934 12284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:00:11.751346 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 12284
I20260620 23:00:11.751559 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:00:11.752238 2996 external_mini_cluster.cc:1428] Setting key f0b62bb7a7fe3b72a6babf8e6801b848
I20260620 23:00:11.752965 12350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:00:11.768175 12350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e93ff2139be43f48287bb6a094b44a9: Bootstrap starting.
I20260620 23:00:11.768467 12353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:11.754261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50230 (local address 127.2.237.62:41787)
0620 23:00:11.755320 (+ 1059us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:11.755337 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:11.755387 (+ 50us) server_negotiation.cc:407] Connection header received
0620 23:00:11.757688 (+ 2301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:11.757717 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:11.758336 (+ 619us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:11.758829 (+ 493us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:11.759375 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:11.761390 (+ 2015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:11.762916 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:11.763979 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:11.764182 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:11.765568 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:00:11.765582 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:00:11.765608 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:00:11.766325 (+ 717us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:00:11.766423 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:00:11.766448 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:00:11.766590 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0620 23:00:11.766852 (+ 262us) server_negotiation.cc:299] Negotiation successful
0620 23:00:11.767176 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":342,"threads_started":1}
I20260620 23:00:11.771900 12350 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e93ff2139be43f48287bb6a094b44a9: Neither blocks nor log segments found. Creating new log.
I20260620 23:00:11.772845 12350 log.cc:824] T 00000000000000000000000000000000 P 3e93ff2139be43f48287bb6a094b44a9: Log is configured to *not* fsync() on all Append() calls
I20260620 23:00:11.775691 12350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e93ff2139be43f48287bb6a094b44a9: No bootstrap required, opened a new log
I20260620 23:00:11.783299 12350 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e93ff2139be43f48287bb6a094b44a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e93ff2139be43f48287bb6a094b44a9" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 41787 } }
I20260620 23:00:11.783651 12350 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e93ff2139be43f48287bb6a094b44a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:00:11.783726 12350 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e93ff2139be43f48287bb6a094b44a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e93ff2139be43f48287bb6a094b44a9, State: Initialized, Role: FOLLOWER
I20260620 23:00:11.784288 12350 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e93ff2139be43f48287bb6a094b44a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e93ff2139be43f48287bb6a094b44a9" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 41787 } }
I20260620 23:00:11.784484 12350 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e93ff2139be43f48287bb6a094b44a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:00:11.784582 12350 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e93ff2139be43f48287bb6a094b44a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:00:11.784703 12350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e93ff2139be43f48287bb6a094b44a9 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:00:11.786810 12350 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e93ff2139be43f48287bb6a094b44a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e93ff2139be43f48287bb6a094b44a9" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 41787 } }
I20260620 23:00:11.787261 12350 leader_election.cc:302] T 00000000000000000000000000000000 P 3e93ff2139be43f48287bb6a094b44a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e93ff2139be43f48287bb6a094b44a9; no voters:
I20260620 23:00:11.787812 12350 leader_election.cc:288] T 00000000000000000000000000000000 P 3e93ff2139be43f48287bb6a094b44a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:00:11.788097 12355 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e93ff2139be43f48287bb6a094b44a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:00:11.788968 12355 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e93ff2139be43f48287bb6a094b44a9 [term 1 LEADER]: Becoming Leader. State: Replica: 3e93ff2139be43f48287bb6a094b44a9, State: Running, Role: LEADER
I20260620 23:00:11.789621 12355 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e93ff2139be43f48287bb6a094b44a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e93ff2139be43f48287bb6a094b44a9" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 41787 } }
I20260620 23:00:11.790055 12350 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e93ff2139be43f48287bb6a094b44a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:00:11.793251 12356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e93ff2139be43f48287bb6a094b44a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e93ff2139be43f48287bb6a094b44a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e93ff2139be43f48287bb6a094b44a9" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 41787 } } }
I20260620 23:00:11.793318 12357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e93ff2139be43f48287bb6a094b44a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e93ff2139be43f48287bb6a094b44a9. Latest consensus state: current_term: 1 leader_uuid: "3e93ff2139be43f48287bb6a094b44a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e93ff2139be43f48287bb6a094b44a9" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 41787 } } }
I20260620 23:00:11.793709 12356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e93ff2139be43f48287bb6a094b44a9 [sys.catalog]: This master's current role is: LEADER
I20260620 23:00:11.793761 12357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e93ff2139be43f48287bb6a094b44a9 [sys.catalog]: This master's current role is: LEADER
I20260620 23:00:11.794720 12362 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:00:11.801210 12362 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:00:11.810915 12362 catalog_manager.cc:1347] Generated new cluster ID: e6c64f79954b49298dad6a6c7118df94
I20260620 23:00:11.811038 12362 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:00:11.843451 12362 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:00:11.845348 12362 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:00:11.857555 12362 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e93ff2139be43f48287bb6a094b44a9: Generated new TSK 0
I20260620 23:00:11.858698 12362 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:00:11.869829 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:0
--local_ip_for_outbound_sockets=127.2.237.1
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:41787
--builtin_ntp_servers=127.2.237.20:41599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260620 23:00:12.207469 12374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:00:12.207755 12374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:00:12.207839 12374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:00:12.215145 12374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:00:12.215257 12374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:00:12.215371 12374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 23:00:12.224673 12374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:41599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:41787
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:00:12.226018 12374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:00:12.227859 12374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:00:12.241219 12379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:00:12.241245 12382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:00:12.241246 12380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:00:12.242185 12374 server_base.cc:1034] running on GCE node
I20260620 23:00:12.242877 12374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:00:12.243803 12374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:00:12.245003 12374 hybrid_clock.cc:648] HybridClock initialized: now 1781996412244962 us; error 41 us; skew 500 ppm
I20260620 23:00:12.245280 12374 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 23:00:12.247550 12374 webserver.cc:460] Webserver started at http://127.2.237.1:43513/ using document root <none> and password file <none>
I20260620 23:00:12.248226 12374 fs_manager.cc:359] Metadata directory not provided
I20260620 23:00:12.248311 12374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:00:12.248598 12374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:00:12.251523 12374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/data/instance:
uuid: "0cd1cfbec92c4320b56e0807bda671ff"
format_stamp: "Formatted at 2026-06-20 23:00:12 on dist-test-slave-f63b"
server_key: "3b4faa4de309dc57e2b7f9aa1b42d7f1"
server_key_iv: "a7b8f9b5e976d0103066403ea9a2c9b5"
server_key_version: "encryptionkey@0"
I20260620 23:00:12.252346 12374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance:
uuid: "0cd1cfbec92c4320b56e0807bda671ff"
format_stamp: "Formatted at 2026-06-20 23:00:12 on dist-test-slave-f63b"
server_key: "3b4faa4de309dc57e2b7f9aa1b42d7f1"
server_key_iv: "a7b8f9b5e976d0103066403ea9a2c9b5"
server_key_version: "encryptionkey@0"
I20260620 23:00:12.257604 12374 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260620 23:00:12.261222 12388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:00:12.262840 12374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 23:00:12.263032 12374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "0cd1cfbec92c4320b56e0807bda671ff"
format_stamp: "Formatted at 2026-06-20 23:00:12 on dist-test-slave-f63b"
server_key: "3b4faa4de309dc57e2b7f9aa1b42d7f1"
server_key_iv: "a7b8f9b5e976d0103066403ea9a2c9b5"
server_key_version: "encryptionkey@0"
I20260620 23:00:12.263278 12374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:00:12.288393 12374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:00:12.289388 12374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:00:12.289711 12374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:00:12.290917 12374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:00:12.292795 12374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:00:12.292929 12374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:00:12.293083 12374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:00:12.293179 12374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:00:12.329610 12374 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:34841
I20260620 23:00:12.329712 12500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:34841 every 8 connection(s)
I20260620 23:00:12.331103 12374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 23:00:12.334287 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 12374
I20260620 23:00:12.334486 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance
I20260620 23:00:12.335376 2996 external_mini_cluster.cc:1428] Setting key 11658067c923f67dc89dd3803168fddb
I20260620 23:00:12.338339 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:0
--local_ip_for_outbound_sockets=127.2.237.2
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:41787
--builtin_ntp_servers=127.2.237.20:41599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260620 23:00:12.345113 12503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:12.333916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:35425 (local address 127.2.237.62:41787)
0620 23:00:12.334435 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:12.334442 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:12.336257 (+ 1815us) server_negotiation.cc:407] Connection header received
0620 23:00:12.338405 (+ 2148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:12.338410 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:12.338499 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:12.338619 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:12.339753 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:12.340502 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:12.342133 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:12.342543 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:12.342736 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:12.344254 (+ 1518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:00:12.344259 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:00:12.344262 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:00:12.344283 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:00:12.344326 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:00:12.344338 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:00:12.344442 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0620 23:00:12.344674 (+ 232us) server_negotiation.cc:299] Negotiation successful
0620 23:00:12.344770 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":267,"threads_started":1}
I20260620 23:00:12.346544 12504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:12.334801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41787 (local address 127.2.237.1:35425)
0620 23:00:12.336043 (+ 1242us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:12.336095 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:12.338114 (+ 2019us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:12.338880 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:12.338896 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:12.338992 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:12.339578 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:12.339596 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:12.340671 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:12.340683 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:12.341980 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:12.341993 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:12.342831 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:12.342839 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:12.343109 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:12.343860 (+ 751us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:00:12.343862 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:12.343932 (+ 70us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:00:12.344038 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:00:12.344443 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:00:12.344455 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:00:12.344466 (+ 11us) client_negotiation.cc:769] Sending connection context
0620 23:00:12.344666 (+ 200us) client_negotiation.cc:240] Negotiation successful
0620 23:00:12.345108 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":342,"threads_started":1}
I20260620 23:00:12.348263 12501 heartbeater.cc:347] Connected to a master server at 127.2.237.62:41787
I20260620 23:00:12.348598 12501 heartbeater.cc:464] Registering TS with master...
I20260620 23:00:12.349545 12501 heartbeater.cc:511] Master 127.2.237.62:41787 requested a full tablet report, sending...
I20260620 23:00:12.351747 12315 ts_manager.cc:194] Registered new tserver with Master: 0cd1cfbec92c4320b56e0807bda671ff (127.2.237.1:34841)
I20260620 23:00:12.353274 12315 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.237.1:35425
W20260620 23:00:12.676571 12505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:00:12.676924 12505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:00:12.677083 12505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:00:12.684522 12505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:00:12.684671 12505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:00:12.684847 12505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 23:00:12.694370 12505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:41599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:41787
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:00:12.695801 12505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:00:12.697686 12505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:00:12.717195 12510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:00:12.717854 12511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:00:12.720638 12505 server_base.cc:1034] running on GCE node
W20260620 23:00:12.721573 12513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:00:12.723337 12505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:00:12.724406 12505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:00:12.725625 12505 hybrid_clock.cc:648] HybridClock initialized: now 1781996412725565 us; error 48 us; skew 500 ppm
I20260620 23:00:12.726009 12505 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 23:00:12.728626 12505 webserver.cc:460] Webserver started at http://127.2.237.2:33425/ using document root <none> and password file <none>
I20260620 23:00:12.729426 12505 fs_manager.cc:359] Metadata directory not provided
I20260620 23:00:12.729521 12505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:00:12.729863 12505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:00:12.732982 12505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/data/instance:
uuid: "8f0265429b3747489fc41f36977a1abc"
format_stamp: "Formatted at 2026-06-20 23:00:12 on dist-test-slave-f63b"
server_key: "eeaeb4ba06ec3a1bccbaa01dfea2e001"
server_key_iv: "92bcfd051a89ab18e4eba273c77f85ad"
server_key_version: "encryptionkey@0"
I20260620 23:00:12.733837 12505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance:
uuid: "8f0265429b3747489fc41f36977a1abc"
format_stamp: "Formatted at 2026-06-20 23:00:12 on dist-test-slave-f63b"
server_key: "eeaeb4ba06ec3a1bccbaa01dfea2e001"
server_key_iv: "92bcfd051a89ab18e4eba273c77f85ad"
server_key_version: "encryptionkey@0"
I20260620 23:00:12.739598 12505 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260620 23:00:12.743336 12519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:00:12.744908 12505 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 23:00:12.745115 12505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "8f0265429b3747489fc41f36977a1abc"
format_stamp: "Formatted at 2026-06-20 23:00:12 on dist-test-slave-f63b"
server_key: "eeaeb4ba06ec3a1bccbaa01dfea2e001"
server_key_iv: "92bcfd051a89ab18e4eba273c77f85ad"
server_key_version: "encryptionkey@0"
I20260620 23:00:12.745360 12505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:00:12.759552 12505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:00:12.760505 12505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:00:12.760823 12505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:00:12.762075 12505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:00:12.763751 12505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:00:12.763842 12505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:00:12.764008 12505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:00:12.764092 12505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:00:12.798461 12505 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:37879
I20260620 23:00:12.798557 12631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:37879 every 8 connection(s)
I20260620 23:00:12.799914 12505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 23:00:12.803556 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 12505
I20260620 23:00:12.803751 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance
I20260620 23:00:12.804255 2996 external_mini_cluster.cc:1428] Setting key c4849e902cc61031e6908a37d488ca2b
I20260620 23:00:12.806885 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:0
--local_ip_for_outbound_sockets=127.2.237.3
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:41787
--builtin_ntp_servers=127.2.237.20:41599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260620 23:00:12.813052 12503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:12.802549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:49279 (local address 127.2.237.62:41787)
0620 23:00:12.802754 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:12.802759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:12.804499 (+ 1740us) server_negotiation.cc:407] Connection header received
0620 23:00:12.806378 (+ 1879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:12.806383 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:12.806467 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:12.806605 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:12.807786 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:12.808523 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:12.810108 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:12.810503 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:12.810676 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:12.812149 (+ 1473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:00:12.812154 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:00:12.812158 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:00:12.812177 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:00:12.812208 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:00:12.812220 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:00:12.812323 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0620 23:00:12.812610 (+ 287us) server_negotiation.cc:299] Negotiation successful
0620 23:00:12.812716 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260620 23:00:12.814153 12634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:12.803139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41787 (local address 127.2.237.2:49279)
0620 23:00:12.804314 (+ 1175us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:12.804364 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:12.806106 (+ 1742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:12.806851 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:12.806860 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:12.806979 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:12.807625 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:12.807643 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:12.808686 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:12.808691 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:12.809960 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:12.809972 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:12.810764 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:12.810771 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:12.811001 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:12.811780 (+ 779us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:00:12.811783 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:12.811835 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:00:12.811919 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:00:12.812378 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:00:12.812387 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:00:12.812393 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 23:00:12.812600 (+ 207us) client_negotiation.cc:240] Negotiation successful
0620 23:00:12.813046 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":311,"threads_started":1}
I20260620 23:00:12.815778 12632 heartbeater.cc:347] Connected to a master server at 127.2.237.62:41787
I20260620 23:00:12.816116 12632 heartbeater.cc:464] Registering TS with master...
I20260620 23:00:12.817092 12632 heartbeater.cc:511] Master 127.2.237.62:41787 requested a full tablet report, sending...
I20260620 23:00:12.818758 12315 ts_manager.cc:194] Registered new tserver with Master: 8f0265429b3747489fc41f36977a1abc (127.2.237.2:37879)
I20260620 23:00:12.819980 12315 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.237.2:49279
W20260620 23:00:13.150871 12635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:00:13.151199 12635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:00:13.151294 12635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:00:13.159415 12635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:00:13.159534 12635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:00:13.159658 12635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 23:00:13.170473 12635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:41599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:41787
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:00:13.171845 12635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:00:13.173755 12635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:00:13.187714 12640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:00:13.187668 12641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:00:13.187747 12643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:00:13.188621 12635 server_base.cc:1034] running on GCE node
I20260620 23:00:13.189249 12635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:00:13.190227 12635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:00:13.191484 12635 hybrid_clock.cc:648] HybridClock initialized: now 1781996413191433 us; error 49 us; skew 500 ppm
I20260620 23:00:13.191879 12635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 23:00:13.194195 12635 webserver.cc:460] Webserver started at http://127.2.237.3:36423/ using document root <none> and password file <none>
I20260620 23:00:13.195019 12635 fs_manager.cc:359] Metadata directory not provided
I20260620 23:00:13.195151 12635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:00:13.195475 12635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:00:13.198341 12635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/data/instance:
uuid: "7ab4b308eb6f49e0b8c693ecc5a827bc"
format_stamp: "Formatted at 2026-06-20 23:00:13 on dist-test-slave-f63b"
server_key: "e7a0c4e6575aa5f9812873d84b126884"
server_key_iv: "b5d350ab54ed6c0997d937a0d624b994"
server_key_version: "encryptionkey@0"
I20260620 23:00:13.199215 12635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance:
uuid: "7ab4b308eb6f49e0b8c693ecc5a827bc"
format_stamp: "Formatted at 2026-06-20 23:00:13 on dist-test-slave-f63b"
server_key: "e7a0c4e6575aa5f9812873d84b126884"
server_key_iv: "b5d350ab54ed6c0997d937a0d624b994"
server_key_version: "encryptionkey@0"
I20260620 23:00:13.204669 12635 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260620 23:00:13.208058 12649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:00:13.209405 12635 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 23:00:13.209589 12635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "7ab4b308eb6f49e0b8c693ecc5a827bc"
format_stamp: "Formatted at 2026-06-20 23:00:13 on dist-test-slave-f63b"
server_key: "e7a0c4e6575aa5f9812873d84b126884"
server_key_iv: "b5d350ab54ed6c0997d937a0d624b994"
server_key_version: "encryptionkey@0"
I20260620 23:00:13.209808 12635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:00:13.227813 12635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:00:13.228750 12635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:00:13.229040 12635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:00:13.230026 12635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:00:13.231552 12635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:00:13.231637 12635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:00:13.231797 12635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:00:13.231895 12635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:00:13.266366 12635 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:44519
I20260620 23:00:13.266399 12761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:44519 every 8 connection(s)
I20260620 23:00:13.267967 12635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 23:00:13.275560 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 12635
I20260620 23:00:13.275739 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance
I20260620 23:00:13.276217 2996 external_mini_cluster.cc:1428] Setting key cd8aeecc7d708fd3ab0259f2613842ae
I20260620 23:00:13.281235 12503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:13.270798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:34705 (local address 127.2.237.62:41787)
0620 23:00:13.271064 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:13.271070 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:13.272544 (+ 1474us) server_negotiation.cc:407] Connection header received
0620 23:00:13.274332 (+ 1788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:13.274337 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:13.274405 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:13.274492 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:13.275885 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:13.276680 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:13.278540 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:13.279100 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:13.279195 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:13.280600 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:00:13.280605 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:00:13.280609 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:00:13.280626 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:00:13.280655 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:00:13.280667 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:00:13.280738 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 23:00:13.280932 (+ 194us) server_negotiation.cc:299] Negotiation successful
0620 23:00:13.281006 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260620 23:00:13.282363 12764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:13.271439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41787 (local address 127.2.237.3:34705)
0620 23:00:13.272365 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:13.272410 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:13.274055 (+ 1645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:13.274854 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:13.274864 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:13.274964 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:13.275665 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:13.275686 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:13.276837 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:13.276842 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:13.278398 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:13.278413 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:13.279373 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:13.279380 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:13.279590 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:13.280249 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:00:13.280252 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:13.280339 (+ 87us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:00:13.280420 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:00:13.280751 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:00:13.280758 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:00:13.280764 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 23:00:13.280909 (+ 145us) client_negotiation.cc:240] Negotiation successful
0620 23:00:13.281217 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":283,"threads_started":1}
I20260620 23:00:13.283921 12762 heartbeater.cc:347] Connected to a master server at 127.2.237.62:41787
I20260620 23:00:13.284256 12762 heartbeater.cc:464] Registering TS with master...
I20260620 23:00:13.285075 12762 heartbeater.cc:511] Master 127.2.237.62:41787 requested a full tablet report, sending...
I20260620 23:00:13.286660 12315 ts_manager.cc:194] Registered new tserver with Master: 7ab4b308eb6f49e0b8c693ecc5a827bc (127.2.237.3:44519)
I20260620 23:00:13.287503 12315 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.237.3:34705
I20260620 23:00:13.291900 2996 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 23:00:13.298259 12503 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:13.296557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50232 (local address 127.2.237.62:41787)
0620 23:00:13.296807 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:13.296815 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:13.297137 (+ 322us) server_negotiation.cc:407] Connection header received
0620 23:00:13.297316 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:13.297321 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:13.297450 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:13.297556 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:13.298036 (+ 480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50232: BlockingRecv error: recv got EOF from 127.0.0.1:50232 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260620 23:00:13.298667 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 12374
I20260620 23:00:13.306931 12771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:13.299616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52340 (local address 127.2.237.1:34841)
0620 23:00:13.300140 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:13.300147 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:13.300172 (+ 25us) server_negotiation.cc:407] Connection header received
0620 23:00:13.300224 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:13.300230 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:13.300448 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:13.300638 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:13.300900 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:13.301870 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:13.303231 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:13.303901 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:13.304009 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:13.305136 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:00:13.305144 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:00:13.305157 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:00:13.305774 (+ 617us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:00:13.305821 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:00:13.305839 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:00:13.305934 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0620 23:00:13.306112 (+ 178us) server_negotiation.cc:299] Negotiation successful
0620 23:00:13.306285 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":276,"threads_started":1}
I20260620 23:00:13.308246 12496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:00:13.390282 12501 heartbeater.cc:503] Master 127.2.237.62:41787 was elected leader, sending a full tablet report...
I20260620 23:00:13.391474 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 12374
W20260620 23:00:13.408183 12302 connection.cc:535] server connection from 127.2.237.1:35425 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 23:00:13.408527 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 12505
I20260620 23:00:13.416407 12773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:13.409394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33756 (local address 127.2.237.2:37879)
0620 23:00:13.409932 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:13.409945 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:13.409973 (+ 28us) server_negotiation.cc:407] Connection header received
0620 23:00:13.410047 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:13.410055 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:13.410293 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:13.410438 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:13.410821 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:13.411859 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:13.413015 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:13.413677 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:13.413853 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:13.414938 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:00:13.414947 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:00:13.414960 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:00:13.415638 (+ 678us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:00:13.415687 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:00:13.415712 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:00:13.415814 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0620 23:00:13.415936 (+ 122us) server_negotiation.cc:299] Negotiation successful
0620 23:00:13.416072 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":282,"threads_started":1}
I20260620 23:00:13.417757 12627 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:00:13.500682 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 12505
I20260620 23:00:13.516898 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 12635
I20260620 23:00:13.525395 12775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:13.517959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45308 (local address 127.2.237.3:44519)
0620 23:00:13.518521 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:13.518529 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:13.518558 (+ 29us) server_negotiation.cc:407] Connection header received
0620 23:00:13.518684 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:13.518691 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:13.518920 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:13.519098 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:13.519577 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:13.520615 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:13.521868 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:13.522448 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:13.522634 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:13.523920 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:00:13.523929 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:00:13.523943 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:00:13.524549 (+ 606us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:00:13.524594 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:00:13.524614 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:00:13.524753 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0620 23:00:13.524903 (+ 150us) server_negotiation.cc:299] Negotiation successful
0620 23:00:13.525034 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":248,"threads_started":1}
I20260620 23:00:13.526937 12757 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:00:13.609812 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 12635
I20260620 23:00:13.626513 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 12284
I20260620 23:00:13.627774 12345 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:00:13.731833 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 12284
2026-06-20T23:00:13Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2497 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-20T23:00:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:00:13Z Disabled control of system clock
I20260620 23:00:13.803925 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:34755
--webserver_interface=127.2.237.62
--webserver_port=0
--builtin_ntp_servers=127.2.237.20:36525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:34755
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260620 23:00:14.134620 12781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:00:14.134939 12781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:00:14.135026 12781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:00:14.142170 12781 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:00:14.142299 12781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:00:14.142336 12781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:00:14.142385 12781 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:00:14.142419 12781 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:00:14.153010 12781 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:36525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:34755
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:34755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:00:14.154459 12781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:00:14.156556 12781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:00:14.169991 12786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:00:14.170037 12789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:00:14.170004 12787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:00:14.176406 12781 server_base.cc:1034] running on GCE node
I20260620 23:00:14.177382 12781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:00:14.179170 12781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:00:14.180421 12781 hybrid_clock.cc:648] HybridClock initialized: now 1781996414180390 us; error 43 us; skew 500 ppm
I20260620 23:00:14.180826 12781 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 23:00:14.183450 12781 webserver.cc:460] Webserver started at http://127.2.237.62:33847/ using document root <none> and password file <none>
I20260620 23:00:14.184239 12781 fs_manager.cc:359] Metadata directory not provided
I20260620 23:00:14.184391 12781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:00:14.184772 12781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:00:14.188056 12781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "7c89b23b787a403fa683113590becb2e"
format_stamp: "Formatted at 2026-06-20 23:00:14 on dist-test-slave-f63b"
server_key: "bae051d111f37f23898cd947582fe978"
server_key_iv: "7330bde6e6478ab067921fc3c2dcc9bf"
server_key_version: "encryptionkey@0"
I20260620 23:00:14.188928 12781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "7c89b23b787a403fa683113590becb2e"
format_stamp: "Formatted at 2026-06-20 23:00:14 on dist-test-slave-f63b"
server_key: "bae051d111f37f23898cd947582fe978"
server_key_iv: "7330bde6e6478ab067921fc3c2dcc9bf"
server_key_version: "encryptionkey@0"
I20260620 23:00:14.194518 12781 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260620 23:00:14.198346 12795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:00:14.199965 12781 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 23:00:14.200181 12781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "7c89b23b787a403fa683113590becb2e"
format_stamp: "Formatted at 2026-06-20 23:00:14 on dist-test-slave-f63b"
server_key: "bae051d111f37f23898cd947582fe978"
server_key_iv: "7330bde6e6478ab067921fc3c2dcc9bf"
server_key_version: "encryptionkey@0"
I20260620 23:00:14.200428 12781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:00:14.207038 12781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:00:14.207942 12781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:00:14.208297 12781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:00:14.228367 12781 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:34755
I20260620 23:00:14.228399 12846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:34755 every 8 connection(s)
I20260620 23:00:14.229821 12781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:00:14.233851 12847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:00:14.236577 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 12781
I20260620 23:00:14.236829 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:00:14.237509 2996 external_mini_cluster.cc:1428] Setting key 90ca7bfb3bd95509a3a6f36d7205c352
I20260620 23:00:14.247463 12850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:14.239827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33086 (local address 127.2.237.62:34755)
0620 23:00:14.240966 (+ 1139us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:14.240981 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:14.241033 (+ 52us) server_negotiation.cc:407] Connection header received
0620 23:00:14.243107 (+ 2074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:14.243144 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:14.243761 (+ 617us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:14.244309 (+ 548us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:14.245416 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:00:14.245424 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:00:14.245443 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:00:14.245460 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:00:14.245543 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:00:14.245578 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:00:14.245801 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0620 23:00:14.245924 (+ 123us) server_negotiation.cc:299] Negotiation successful
0620 23:00:14.246196 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":370,"threads_started":1}
I20260620 23:00:14.247905 12847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c89b23b787a403fa683113590becb2e: Bootstrap starting.
I20260620 23:00:14.251050 12847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c89b23b787a403fa683113590becb2e: Neither blocks nor log segments found. Creating new log.
I20260620 23:00:14.252043 12847 log.cc:824] T 00000000000000000000000000000000 P 7c89b23b787a403fa683113590becb2e: Log is configured to *not* fsync() on all Append() calls
I20260620 23:00:14.254850 12847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c89b23b787a403fa683113590becb2e: No bootstrap required, opened a new log
I20260620 23:00:14.261900 12847 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c89b23b787a403fa683113590becb2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c89b23b787a403fa683113590becb2e" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 34755 } }
I20260620 23:00:14.262305 12847 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c89b23b787a403fa683113590becb2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:00:14.262413 12847 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c89b23b787a403fa683113590becb2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c89b23b787a403fa683113590becb2e, State: Initialized, Role: FOLLOWER
I20260620 23:00:14.263017 12847 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c89b23b787a403fa683113590becb2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c89b23b787a403fa683113590becb2e" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 34755 } }
I20260620 23:00:14.263262 12847 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c89b23b787a403fa683113590becb2e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:00:14.263409 12847 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c89b23b787a403fa683113590becb2e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:00:14.263585 12847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c89b23b787a403fa683113590becb2e [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:00:14.265800 12847 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c89b23b787a403fa683113590becb2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c89b23b787a403fa683113590becb2e" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 34755 } }
I20260620 23:00:14.266325 12847 leader_election.cc:302] T 00000000000000000000000000000000 P 7c89b23b787a403fa683113590becb2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c89b23b787a403fa683113590becb2e; no voters:
I20260620 23:00:14.266875 12847 leader_election.cc:288] T 00000000000000000000000000000000 P 7c89b23b787a403fa683113590becb2e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:00:14.267010 12852 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c89b23b787a403fa683113590becb2e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:00:14.267742 12852 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c89b23b787a403fa683113590becb2e [term 1 LEADER]: Becoming Leader. State: Replica: 7c89b23b787a403fa683113590becb2e, State: Running, Role: LEADER
I20260620 23:00:14.268321 12852 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c89b23b787a403fa683113590becb2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c89b23b787a403fa683113590becb2e" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 34755 } }
I20260620 23:00:14.269023 12847 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c89b23b787a403fa683113590becb2e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:00:14.271373 12854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c89b23b787a403fa683113590becb2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c89b23b787a403fa683113590becb2e. Latest consensus state: current_term: 1 leader_uuid: "7c89b23b787a403fa683113590becb2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c89b23b787a403fa683113590becb2e" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 34755 } } }
I20260620 23:00:14.271418 12853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c89b23b787a403fa683113590becb2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c89b23b787a403fa683113590becb2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c89b23b787a403fa683113590becb2e" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 34755 } } }
I20260620 23:00:14.271798 12854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c89b23b787a403fa683113590becb2e [sys.catalog]: This master's current role is: LEADER
I20260620 23:00:14.271806 12853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c89b23b787a403fa683113590becb2e [sys.catalog]: This master's current role is: LEADER
I20260620 23:00:14.277881 12859 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:00:14.283550 12859 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:00:14.294430 12859 catalog_manager.cc:1347] Generated new cluster ID: 709b518f153646b584fa2c54b5bfdb1d
I20260620 23:00:14.294622 12859 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:00:14.319195 12859 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:00:14.319710 12859 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:00:14.330641 12859 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c89b23b787a403fa683113590becb2e: Generated new TSK 0
I20260620 23:00:14.331852 12859 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:00:14.348153 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:0
--local_ip_for_outbound_sockets=127.2.237.1
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:34755
--builtin_ntp_servers=127.2.237.20:36525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260620 23:00:14.690331 12871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:00:14.690817 12871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:00:14.691048 12871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:00:14.698635 12871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:00:14.698769 12871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:00:14.698977 12871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 23:00:14.713413 12871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:36525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:34755
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:00:14.714926 12871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:00:14.716832 12871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:00:14.730875 12877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:00:14.730898 12876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:00:14.730926 12879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:00:14.732185 12871 server_base.cc:1034] running on GCE node
I20260620 23:00:14.732975 12871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:00:14.733991 12871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:00:14.735241 12871 hybrid_clock.cc:648] HybridClock initialized: now 1781996414735192 us; error 43 us; skew 500 ppm
I20260620 23:00:14.735636 12871 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 23:00:14.738029 12871 webserver.cc:460] Webserver started at http://127.2.237.1:37467/ using document root <none> and password file <none>
I20260620 23:00:14.738863 12871 fs_manager.cc:359] Metadata directory not provided
I20260620 23:00:14.739001 12871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:00:14.739341 12871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:00:14.742336 12871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/data/instance:
uuid: "1b95657e075b4c48a44c87d753b26f2f"
format_stamp: "Formatted at 2026-06-20 23:00:14 on dist-test-slave-f63b"
server_key: "49640304ec1da273fd8ae0bc33aa1032"
server_key_iv: "162c31de0d753140c12faed7dc9bea8e"
server_key_version: "encryptionkey@0"
I20260620 23:00:14.743269 12871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance:
uuid: "1b95657e075b4c48a44c87d753b26f2f"
format_stamp: "Formatted at 2026-06-20 23:00:14 on dist-test-slave-f63b"
server_key: "49640304ec1da273fd8ae0bc33aa1032"
server_key_iv: "162c31de0d753140c12faed7dc9bea8e"
server_key_version: "encryptionkey@0"
I20260620 23:00:14.749243 12871 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260620 23:00:14.753003 12885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:00:14.754520 12871 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260620 23:00:14.754730 12871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "1b95657e075b4c48a44c87d753b26f2f"
format_stamp: "Formatted at 2026-06-20 23:00:14 on dist-test-slave-f63b"
server_key: "49640304ec1da273fd8ae0bc33aa1032"
server_key_iv: "162c31de0d753140c12faed7dc9bea8e"
server_key_version: "encryptionkey@0"
I20260620 23:00:14.754952 12871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:00:14.760859 12871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:00:14.761646 12871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:00:14.761886 12871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:00:14.762830 12871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:00:14.764251 12871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:00:14.764339 12871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:00:14.764426 12871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:00:14.764480 12871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:00:14.798026 12871 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:44877
I20260620 23:00:14.798079 12997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:44877 every 8 connection(s)
I20260620 23:00:14.799495 12871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 23:00:14.801527 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 12871
I20260620 23:00:14.801817 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance
I20260620 23:00:14.802732 2996 external_mini_cluster.cc:1428] Setting key 634e292ec6378859d7a0ca9619803a18
I20260620 23:00:14.806074 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:0
--local_ip_for_outbound_sockets=127.2.237.2
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:34755
--builtin_ntp_servers=127.2.237.20:36525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260620 23:00:14.809167 13000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:14.802409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:58687 (local address 127.2.237.62:34755)
0620 23:00:14.802905 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:14.802910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:14.804613 (+ 1703us) server_negotiation.cc:407] Connection header received
0620 23:00:14.806607 (+ 1994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:14.806617 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:14.806694 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:14.806833 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:14.808238 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:00:14.808247 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:00:14.808253 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:00:14.808279 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:00:14.808338 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:00:14.808358 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:00:14.808508 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0620 23:00:14.808677 (+ 169us) server_negotiation.cc:299] Negotiation successful
0620 23:00:14.808758 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":249,"threads_started":1}
I20260620 23:00:14.809983 13001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:14.803229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:34755 (local address 127.2.237.1:58687)
0620 23:00:14.804419 (+ 1190us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:14.804474 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:14.806252 (+ 1778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:14.807059 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:14.807068 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:14.807189 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:14.807900 (+ 711us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:00:14.807902 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:14.807942 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:00:14.808047 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:00:14.808468 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:00:14.808475 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:00:14.808491 (+ 16us) client_negotiation.cc:769] Sending connection context
0620 23:00:14.808708 (+ 217us) client_negotiation.cc:240] Negotiation successful
0620 23:00:14.808931 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":367,"threads_started":1}
I20260620 23:00:14.811638 12998 heartbeater.cc:347] Connected to a master server at 127.2.237.62:34755
I20260620 23:00:14.811972 12998 heartbeater.cc:464] Registering TS with master...
I20260620 23:00:14.812661 12998 heartbeater.cc:511] Master 127.2.237.62:34755 requested a full tablet report, sending...
I20260620 23:00:14.815179 12812 ts_manager.cc:194] Registered new tserver with Master: 1b95657e075b4c48a44c87d753b26f2f (127.2.237.1:44877)
W20260620 23:00:15.141695 13002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:00:15.141995 13002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:00:15.142148 13002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:00:15.149506 13002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:00:15.149609 13002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:00:15.149832 13002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 23:00:15.159396 13002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:36525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:34755
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:00:15.160974 13002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:00:15.162842 13002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:00:15.176070 13007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:00:15.176151 13008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:00:15.176110 13010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:00:15.177361 13002 server_base.cc:1034] running on GCE node
I20260620 23:00:15.178000 13002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:00:15.179133 13002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:00:15.180361 13002 hybrid_clock.cc:648] HybridClock initialized: now 1781996415180296 us; error 63 us; skew 500 ppm
I20260620 23:00:15.180694 13002 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 23:00:15.183082 13002 webserver.cc:460] Webserver started at http://127.2.237.2:43385/ using document root <none> and password file <none>
I20260620 23:00:15.183805 13002 fs_manager.cc:359] Metadata directory not provided
I20260620 23:00:15.183934 13002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:00:15.184270 13002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:00:15.188608 13002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/data/instance:
uuid: "1ea239311e4d4c60918a32913ef72a4a"
format_stamp: "Formatted at 2026-06-20 23:00:15 on dist-test-slave-f63b"
server_key: "fa06db135fab70bb5728d718e1993fc1"
server_key_iv: "c27e2eed00ef289d0991ffbaaa5057e3"
server_key_version: "encryptionkey@0"
I20260620 23:00:15.189436 13002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance:
uuid: "1ea239311e4d4c60918a32913ef72a4a"
format_stamp: "Formatted at 2026-06-20 23:00:15 on dist-test-slave-f63b"
server_key: "fa06db135fab70bb5728d718e1993fc1"
server_key_iv: "c27e2eed00ef289d0991ffbaaa5057e3"
server_key_version: "encryptionkey@0"
I20260620 23:00:15.194849 13002 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260620 23:00:15.199158 13016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:00:15.200693 13002 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260620 23:00:15.200881 13002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "1ea239311e4d4c60918a32913ef72a4a"
format_stamp: "Formatted at 2026-06-20 23:00:15 on dist-test-slave-f63b"
server_key: "fa06db135fab70bb5728d718e1993fc1"
server_key_iv: "c27e2eed00ef289d0991ffbaaa5057e3"
server_key_version: "encryptionkey@0"
I20260620 23:00:15.201123 13002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:00:15.207320 13002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:00:15.208139 13002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:00:15.208410 13002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:00:15.209342 13002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:00:15.210863 13002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:00:15.210996 13002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:00:15.211124 13002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:00:15.211213 13002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:00:15.246188 13002 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:39055
I20260620 23:00:15.246223 13128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:39055 every 8 connection(s)
I20260620 23:00:15.247565 13002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 23:00:15.251506 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 13002
I20260620 23:00:15.251725 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance
I20260620 23:00:15.252341 2996 external_mini_cluster.cc:1428] Setting key d02cf13975815a917d02fd32cbb315eb
I20260620 23:00:15.255508 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:0
--local_ip_for_outbound_sockets=127.2.237.3
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:34755
--builtin_ntp_servers=127.2.237.20:36525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260620 23:00:15.256968 13000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:15.250297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:36477 (local address 127.2.237.62:34755)
0620 23:00:15.250561 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:15.250607 (+ 46us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:15.252339 (+ 1732us) server_negotiation.cc:407] Connection header received
0620 23:00:15.254642 (+ 2303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:15.254647 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:15.254721 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:15.254834 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:15.256200 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:00:15.256205 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:00:15.256210 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:00:15.256229 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:00:15.256271 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:00:15.256283 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:00:15.256434 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0620 23:00:15.256645 (+ 211us) server_negotiation.cc:299] Negotiation successful
0620 23:00:15.256712 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
I20260620 23:00:15.257997 13131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:15.250896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:34755 (local address 127.2.237.2:36477)
0620 23:00:15.252125 (+ 1229us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:15.252184 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:15.254282 (+ 2098us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:15.255096 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:15.255105 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:15.255207 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:15.255893 (+ 686us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:00:15.255895 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:15.255935 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:00:15.256031 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:00:15.256418 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:00:15.256429 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:00:15.256437 (+ 8us) client_negotiation.cc:769] Sending connection context
0620 23:00:15.256737 (+ 300us) client_negotiation.cc:240] Negotiation successful
0620 23:00:15.257035 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":344,"threads_started":1}
I20260620 23:00:15.259456 13129 heartbeater.cc:347] Connected to a master server at 127.2.237.62:34755
I20260620 23:00:15.259785 13129 heartbeater.cc:464] Registering TS with master...
I20260620 23:00:15.260396 13129 heartbeater.cc:511] Master 127.2.237.62:34755 requested a full tablet report, sending...
I20260620 23:00:15.261963 12812 ts_manager.cc:194] Registered new tserver with Master: 1ea239311e4d4c60918a32913ef72a4a (127.2.237.2:39055)
W20260620 23:00:15.591464 13132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:00:15.591812 13132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:00:15.591984 13132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:00:15.599447 13132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:00:15.599596 13132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:00:15.599789 13132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 23:00:15.609422 13132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:36525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:34755
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:00:15.610827 13132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:00:15.612591 13132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:00:15.625810 13138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:00:15.625906 13137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:00:15.625887 13140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:00:15.626808 13132 server_base.cc:1034] running on GCE node
I20260620 23:00:15.627475 13132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:00:15.628567 13132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:00:15.629773 13132 hybrid_clock.cc:648] HybridClock initialized: now 1781996415629719 us; error 43 us; skew 500 ppm
I20260620 23:00:15.630134 13132 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 23:00:15.636284 13132 webserver.cc:460] Webserver started at http://127.2.237.3:44617/ using document root <none> and password file <none>
I20260620 23:00:15.637074 13132 fs_manager.cc:359] Metadata directory not provided
I20260620 23:00:15.637203 13132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:00:15.637558 13132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:00:15.640558 13132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/data/instance:
uuid: "64b0fc9809604f8a8fe8c59efd214f2c"
format_stamp: "Formatted at 2026-06-20 23:00:15 on dist-test-slave-f63b"
server_key: "24770defafc542727d56f9917189e41e"
server_key_iv: "d478119c3253915d8e3032edebdf33cc"
server_key_version: "encryptionkey@0"
I20260620 23:00:15.641358 13132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance:
uuid: "64b0fc9809604f8a8fe8c59efd214f2c"
format_stamp: "Formatted at 2026-06-20 23:00:15 on dist-test-slave-f63b"
server_key: "24770defafc542727d56f9917189e41e"
server_key_iv: "d478119c3253915d8e3032edebdf33cc"
server_key_version: "encryptionkey@0"
I20260620 23:00:15.646699 13132 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260620 23:00:15.650249 13146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:00:15.651873 13132 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260620 23:00:15.652074 13132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "64b0fc9809604f8a8fe8c59efd214f2c"
format_stamp: "Formatted at 2026-06-20 23:00:15 on dist-test-slave-f63b"
server_key: "24770defafc542727d56f9917189e41e"
server_key_iv: "d478119c3253915d8e3032edebdf33cc"
server_key_version: "encryptionkey@0"
I20260620 23:00:15.652359 13132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:00:15.658313 13132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:00:15.659359 13132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:00:15.659643 13132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:00:15.660604 13132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:00:15.662101 13132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:00:15.662185 13132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:00:15.662366 13132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:00:15.662449 13132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:00:15.695636 13132 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:39211
I20260620 23:00:15.695685 13258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:39211 every 8 connection(s)
I20260620 23:00:15.697108 13132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 23:00:15.700019 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 13132
I20260620 23:00:15.700239 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance
I20260620 23:00:15.700794 2996 external_mini_cluster.cc:1428] Setting key 0e5d27c585ef6858577cd3bb5ba3ce34
I20260620 23:00:15.706296 13000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:15.699894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:53043 (local address 127.2.237.62:34755)
0620 23:00:15.700153 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:15.700160 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:15.701781 (+ 1621us) server_negotiation.cc:407] Connection header received
0620 23:00:15.703786 (+ 2005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:15.703790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:15.703850 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:15.703977 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:15.705493 (+ 1516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:00:15.705499 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:00:15.705505 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:00:15.705524 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:00:15.705565 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:00:15.705578 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:00:15.705704 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0620 23:00:15.705938 (+ 234us) server_negotiation.cc:299] Negotiation successful
0620 23:00:15.705998 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260620 23:00:15.707369 13261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:15.700492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:34755 (local address 127.2.237.3:53043)
0620 23:00:15.701606 (+ 1114us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:15.701653 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:15.703493 (+ 1840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:15.704262 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:15.704271 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:15.704386 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:15.705164 (+ 778us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:00:15.705167 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:15.705208 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:00:15.705305 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:00:15.705678 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:00:15.705689 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:00:15.705698 (+ 9us) client_negotiation.cc:769] Sending connection context
0620 23:00:15.706008 (+ 310us) client_negotiation.cc:240] Negotiation successful
0620 23:00:15.706280 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":344,"threads_started":1}
I20260620 23:00:15.708823 13259 heartbeater.cc:347] Connected to a master server at 127.2.237.62:34755
I20260620 23:00:15.709136 13259 heartbeater.cc:464] Registering TS with master...
I20260620 23:00:15.709757 13259 heartbeater.cc:511] Master 127.2.237.62:34755 requested a full tablet report, sending...
I20260620 23:00:15.711277 12812 ts_manager.cc:194] Registered new tserver with Master: 64b0fc9809604f8a8fe8c59efd214f2c (127.2.237.3:39211)
I20260620 23:00:15.716430 2996 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260620 23:00:15.722533 13000 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:00:15.720914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33096 (local address 127.2.237.62:34755)
0620 23:00:15.721128 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:15.721137 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:15.721631 (+ 494us) server_negotiation.cc:407] Connection header received
0620 23:00:15.721756 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:15.721760 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:15.721822 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:15.721912 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:15.722262 (+ 350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33096: BlockingRecv error: recv got EOF from 127.0.0.1:33096 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260620 23:00:15.722944 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 12871
I20260620 23:00:15.726989 13268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:15.724123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43814 (local address 127.2.237.1:44877)
0620 23:00:15.724753 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:15.724761 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:15.724789 (+ 28us) server_negotiation.cc:407] Connection header received
0620 23:00:15.724860 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:15.724867 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:15.725099 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:15.725236 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:15.726195 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:00:15.726203 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:00:15.726216 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:00:15.726241 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:00:15.726292 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:00:15.726308 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:00:15.726441 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0620 23:00:15.726619 (+ 178us) server_negotiation.cc:299] Negotiation successful
0620 23:00:15.726721 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":324,"threads_started":1}
I20260620 23:00:15.728354 12993 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:00:15.812037 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 12871
I20260620 23:00:15.829465 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 13002
I20260620 23:00:15.833469 13270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:15.830525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49274 (local address 127.2.237.2:39055)
0620 23:00:15.831235 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:15.831242 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:15.831270 (+ 28us) server_negotiation.cc:407] Connection header received
0620 23:00:15.831330 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:15.831337 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:15.831612 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:15.831783 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:15.832772 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:00:15.832778 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:00:15.832791 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:00:15.832803 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:00:15.832845 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:00:15.832859 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:00:15.833012 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0620 23:00:15.833113 (+ 101us) server_negotiation.cc:299] Negotiation successful
0620 23:00:15.833224 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":214,"threads_started":1}
I20260620 23:00:15.834901 13124 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:00:15.917380 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 13002
I20260620 23:00:15.933589 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 13132
I20260620 23:00:15.937525 13272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:15.934773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39324 (local address 127.2.237.3:39211)
0620 23:00:15.935420 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:15.935435 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:15.935466 (+ 31us) server_negotiation.cc:407] Connection header received
0620 23:00:15.935530 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:15.935537 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:15.935759 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:15.935912 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:15.936800 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:00:15.936807 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:00:15.936818 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:00:15.936833 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:00:15.936884 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:00:15.936898 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:00:15.937028 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0620 23:00:15.937086 (+ 58us) server_negotiation.cc:299] Negotiation successful
0620 23:00:15.937205 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":328,"threads_started":1}
I20260620 23:00:15.938920 13254 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:00:16.021471 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 13132
I20260620 23:00:16.037462 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 12781
I20260620 23:00:16.038806 12842 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:00:16.141038 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 12781
2026-06-20T23:00:16Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2409 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:00:16 dist-test-slave-f63b krb5kdc[13277](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:00:16 dist-test-slave-f63b krb5kdc[13277](info): set up 2 sockets
Jun 20 23:00:16 dist-test-slave-f63b krb5kdc[13277](info): commencing operation
krb5kdc: starting...
W20260620 23:00:17.263834 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 23:00:17 dist-test-slave-f63b krb5kdc[13277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:00:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:00:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
I20260620 23:00:17.496727 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:33459
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:43369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:33459
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:00:17.825388 13290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:00:17.825699 13290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:00:17.825788 13290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:00:17.832938 13290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:00:17.833058 13290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:00:17.833106 13290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:00:17.833169 13290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:00:17.833214 13290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:00:17.842971 13290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:43369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:33459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:33459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:00:17.844339 13290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:00:17.845623 13290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:00:17.857312 13295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:00:17.857250 13296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:00:17.857338 13298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:00:17.857997 13290 server_base.cc:1034] running on GCE node
I20260620 23:00:17.858882 13290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:00:17.860270 13290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:00:17.861505 13290 hybrid_clock.cc:648] HybridClock initialized: now 1781996417861458 us; error 36 us; skew 500 ppm
I20260620 23:00:17.861864 13290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:00:17 dist-test-slave-f63b krb5kdc[13277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996417, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:00:17.865008 13290 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 23:00:17.866466 13290 webserver.cc:460] Webserver started at http://127.2.237.62:40523/ using document root <none> and password file <none>
I20260620 23:00:17.867172 13290 fs_manager.cc:359] Metadata directory not provided
I20260620 23:00:17.867272 13290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:00:17.867518 13290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:00:17.870381 13290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "b76afa314106411492df21aaf6cdf7a7"
format_stamp: "Formatted at 2026-06-20 23:00:17 on dist-test-slave-f63b"
server_key: "1f284b362e8e5c34d17c94bfffec2c2e"
server_key_iv: "b77d9cdbdfafe672fd4efa25820fac6c"
server_key_version: "encryptionkey@0"
I20260620 23:00:17.871194 13290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "b76afa314106411492df21aaf6cdf7a7"
format_stamp: "Formatted at 2026-06-20 23:00:17 on dist-test-slave-f63b"
server_key: "1f284b362e8e5c34d17c94bfffec2c2e"
server_key_iv: "b77d9cdbdfafe672fd4efa25820fac6c"
server_key_version: "encryptionkey@0"
I20260620 23:00:17.876549 13290 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260620 23:00:17.880023 13305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:00:17.881503 13290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 23:00:17.881650 13290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "b76afa314106411492df21aaf6cdf7a7"
format_stamp: "Formatted at 2026-06-20 23:00:17 on dist-test-slave-f63b"
server_key: "1f284b362e8e5c34d17c94bfffec2c2e"
server_key_iv: "b77d9cdbdfafe672fd4efa25820fac6c"
server_key_version: "encryptionkey@0"
I20260620 23:00:17.881816 13290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:00:17.900934 13290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:00:17.905596 13290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:00:17.905849 13290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:00:17.924928 13290 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:33459
I20260620 23:00:17.924958 13356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:33459 every 8 connection(s)
I20260620 23:00:17.926399 13290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:00:17.929551 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 13290
I20260620 23:00:17.929770 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:00:17.930454 2996 external_mini_cluster.cc:1428] Setting key 3502611c04a4761efb56be95d5c60604
I20260620 23:00:17.930459 13357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:00:17.945056 13357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b76afa314106411492df21aaf6cdf7a7: Bootstrap starting.
Jun 20 23:00:17 dist-test-slave-f63b krb5kdc[13277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:00:17.948724 13357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b76afa314106411492df21aaf6cdf7a7: Neither blocks nor log segments found. Creating new log.
I20260620 23:00:17.949818 13357 log.cc:824] T 00000000000000000000000000000000 P b76afa314106411492df21aaf6cdf7a7: Log is configured to *not* fsync() on all Append() calls
I20260620 23:00:17.953418 13357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b76afa314106411492df21aaf6cdf7a7: No bootstrap required, opened a new log
I20260620 23:00:17.954689 13360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:17.932453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41706 (local address 127.2.237.62:33459)
0620 23:00:17.933267 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:17.933283 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:17.933339 (+ 56us) server_negotiation.cc:407] Connection header received
0620 23:00:17.935472 (+ 2133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:17.935502 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:17.936042 (+ 540us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:17.936532 (+ 490us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:17.937867 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:17.939872 (+ 2005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:17.941148 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:17.942289 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:17.942491 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:17.946842 (+ 4351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:00:17.946854 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:00:17.946870 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:00:17.946918 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:00:17.950088 (+ 3170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:00:17.950921 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:00:17.950931 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:00:17.950942 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:00:17.951163 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:00:17.951623 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:00:17.951626 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:00:17.951633 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:00:17.952189 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:00:17.952459 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0620 23:00:17.952864 (+ 405us) server_negotiation.cc:299] Negotiation successful
0620 23:00:17.953385 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":322,"threads_started":1}
I20260620 23:00:17.962879 13357 raft_consensus.cc:348] T 00000000000000000000000000000000 P b76afa314106411492df21aaf6cdf7a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b76afa314106411492df21aaf6cdf7a7" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 33459 } }
I20260620 23:00:17.963282 13357 raft_consensus.cc:374] T 00000000000000000000000000000000 P b76afa314106411492df21aaf6cdf7a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:00:17.963413 13357 raft_consensus.cc:729] T 00000000000000000000000000000000 P b76afa314106411492df21aaf6cdf7a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b76afa314106411492df21aaf6cdf7a7, State: Initialized, Role: FOLLOWER
I20260620 23:00:17.964078 13357 consensus_queue.cc:260] T 00000000000000000000000000000000 P b76afa314106411492df21aaf6cdf7a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b76afa314106411492df21aaf6cdf7a7" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 33459 } }
I20260620 23:00:17.964386 13357 raft_consensus.cc:388] T 00000000000000000000000000000000 P b76afa314106411492df21aaf6cdf7a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:00:17.964573 13357 raft_consensus.cc:482] T 00000000000000000000000000000000 P b76afa314106411492df21aaf6cdf7a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:00:17.964772 13357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b76afa314106411492df21aaf6cdf7a7 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:00:17.967119 13357 raft_consensus.cc:504] T 00000000000000000000000000000000 P b76afa314106411492df21aaf6cdf7a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b76afa314106411492df21aaf6cdf7a7" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 33459 } }
I20260620 23:00:17.967585 13357 leader_election.cc:302] T 00000000000000000000000000000000 P b76afa314106411492df21aaf6cdf7a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b76afa314106411492df21aaf6cdf7a7; no voters:
I20260620 23:00:17.968307 13357 leader_election.cc:288] T 00000000000000000000000000000000 P b76afa314106411492df21aaf6cdf7a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:00:17.968421 13362 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b76afa314106411492df21aaf6cdf7a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:00:17.969815 13362 raft_consensus.cc:686] T 00000000000000000000000000000000 P b76afa314106411492df21aaf6cdf7a7 [term 1 LEADER]: Becoming Leader. State: Replica: b76afa314106411492df21aaf6cdf7a7, State: Running, Role: LEADER
I20260620 23:00:17.970315 13362 consensus_queue.cc:237] T 00000000000000000000000000000000 P b76afa314106411492df21aaf6cdf7a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b76afa314106411492df21aaf6cdf7a7" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 33459 } }
I20260620 23:00:17.970614 13357 sys_catalog.cc:564] T 00000000000000000000000000000000 P b76afa314106411492df21aaf6cdf7a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:00:17.973461 13364 sys_catalog.cc:455] T 00000000000000000000000000000000 P b76afa314106411492df21aaf6cdf7a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b76afa314106411492df21aaf6cdf7a7. Latest consensus state: current_term: 1 leader_uuid: "b76afa314106411492df21aaf6cdf7a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b76afa314106411492df21aaf6cdf7a7" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 33459 } } }
I20260620 23:00:17.973852 13364 sys_catalog.cc:458] T 00000000000000000000000000000000 P b76afa314106411492df21aaf6cdf7a7 [sys.catalog]: This master's current role is: LEADER
I20260620 23:00:17.973624 13363 sys_catalog.cc:455] T 00000000000000000000000000000000 P b76afa314106411492df21aaf6cdf7a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b76afa314106411492df21aaf6cdf7a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b76afa314106411492df21aaf6cdf7a7" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 33459 } } }
I20260620 23:00:17.974022 13363 sys_catalog.cc:458] T 00000000000000000000000000000000 P b76afa314106411492df21aaf6cdf7a7 [sys.catalog]: This master's current role is: LEADER
I20260620 23:00:17.980058 13371 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:00:17.986003 13371 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:00:17.994851 13371 catalog_manager.cc:1347] Generated new cluster ID: 38a14024edd7466dbe174aba0456e76e
I20260620 23:00:17.995002 13371 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:00:18.020712 13371 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:00:18.022141 13371 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:00:18.032955 13371 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b76afa314106411492df21aaf6cdf7a7: Generated new TSK 0
I20260620 23:00:18.033936 13371 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.237.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
I20260620 23:00:18.086752 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:0
--local_ip_for_outbound_sockets=127.2.237.1
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:33459
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:43369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:00:18.443089 13383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:00:18.443383 13383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:00:18.443557 13383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:00:18.450919 13383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:00:18.451045 13383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:00:18.451222 13383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 23:00:18.461716 13383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:43369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:33459
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:00:18.463135 13383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:00:18.464457 13383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:00:18.478029 13388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:00:18.477991 13389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:00:18.478067 13391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:00:18.478900 13383 server_base.cc:1034] running on GCE node
I20260620 23:00:18.479574 13383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:00:18.480593 13383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:00:18.481789 13383 hybrid_clock.cc:648] HybridClock initialized: now 1781996418481744 us; error 36 us; skew 500 ppm
I20260620 23:00:18.482131 13383 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:00:18 dist-test-slave-f63b krb5kdc[13277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996418, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:00:18.485301 13383 init.cc:377] Logged in from keytab as kudu/127.2.237.1@KRBTEST.COM (short username kudu)
I20260620 23:00:18.487047 13383 webserver.cc:460] Webserver started at http://127.2.237.1:40481/ using document root <none> and password file <none>
I20260620 23:00:18.487720 13383 fs_manager.cc:359] Metadata directory not provided
I20260620 23:00:18.487820 13383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:00:18.488080 13383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:00:18.491040 13383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/data/instance:
uuid: "110aa0bbf1064e38adb996708256e7c8"
format_stamp: "Formatted at 2026-06-20 23:00:18 on dist-test-slave-f63b"
server_key: "8396b7925cdcf1921de1985188aa9e37"
server_key_iv: "3b03a75a2ffb34efdc7821b3bc3ec792"
server_key_version: "encryptionkey@0"
I20260620 23:00:18.491775 13383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance:
uuid: "110aa0bbf1064e38adb996708256e7c8"
format_stamp: "Formatted at 2026-06-20 23:00:18 on dist-test-slave-f63b"
server_key: "8396b7925cdcf1921de1985188aa9e37"
server_key_iv: "3b03a75a2ffb34efdc7821b3bc3ec792"
server_key_version: "encryptionkey@0"
I20260620 23:00:18.496902 13383 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260620 23:00:18.501523 13398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:00:18.502983 13383 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260620 23:00:18.503209 13383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "110aa0bbf1064e38adb996708256e7c8"
format_stamp: "Formatted at 2026-06-20 23:00:18 on dist-test-slave-f63b"
server_key: "8396b7925cdcf1921de1985188aa9e37"
server_key_iv: "3b03a75a2ffb34efdc7821b3bc3ec792"
server_key_version: "encryptionkey@0"
I20260620 23:00:18.503386 13383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:00:18.518435 13383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:00:18.522703 13383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:00:18.522995 13383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:00:18.523998 13383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:00:18.525305 13383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:00:18.525378 13383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:00:18.525552 13383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:00:18.525621 13383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:00:18.559330 13383 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:45781
I20260620 23:00:18.559382 13510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:45781 every 8 connection(s)
I20260620 23:00:18.560710 13383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 23:00:18.564096 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 13383
I20260620 23:00:18.564327 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance
I20260620 23:00:18.565184 2996 external_mini_cluster.cc:1428] Setting key a9bc9db876f6dbb837cbb27ba280b41d
Jun 20 23:00:18 dist-test-slave-f63b krb5kdc[13277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996418, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:00:18.584777 13513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:18.563561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:58285 (local address 127.2.237.62:33459)
0620 23:00:18.564099 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:18.564107 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:18.565289 (+ 1182us) server_negotiation.cc:407] Connection header received
0620 23:00:18.567544 (+ 2255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:18.567548 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:18.567608 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:18.567699 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:18.569869 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:18.570716 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:18.573280 (+ 2564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:18.573987 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:18.574123 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:18.578124 (+ 4001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:00:18.578129 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:00:18.578132 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:00:18.578165 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:00:18.581127 (+ 2962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:00:18.582206 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:00:18.582210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:00:18.582213 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:00:18.582378 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:00:18.582919 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:00:18.582926 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:00:18.582930 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:00:18.583200 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:00:18.583355 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0620 23:00:18.584162 (+ 807us) server_negotiation.cc:299] Negotiation successful
0620 23:00:18.584387 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":263,"threads_started":1}
I20260620 23:00:18.585901 13514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:18.564090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:33459 (local address 127.2.237.1:58285)
0620 23:00:18.565041 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:18.565093 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:18.567231 (+ 2138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:18.568020 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:18.568035 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:18.568839 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:18.569685 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:18.569712 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:18.570902 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:18.570910 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:18.573089 (+ 2179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:18.573106 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:18.574292 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:18.574300 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:18.574682 (+ 382us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:18.575405 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:18.575428 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:18.577876 (+ 2448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:00:18.581331 (+ 3455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:00:18.581344 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:00:18.581367 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:00:18.582078 (+ 711us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:00:18.582525 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:00:18.582531 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:00:18.582535 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:00:18.582796 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:00:18.583383 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:00:18.583393 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:00:18.583775 (+ 382us) client_negotiation.cc:769] Sending connection context
0620 23:00:18.584057 (+ 282us) client_negotiation.cc:240] Negotiation successful
0620 23:00:18.584630 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":375,"threads_started":1}
I20260620 23:00:18.587664 13511 heartbeater.cc:347] Connected to a master server at 127.2.237.62:33459
I20260620 23:00:18.588061 13511 heartbeater.cc:464] Registering TS with master...
I20260620 23:00:18.589062 13511 heartbeater.cc:511] Master 127.2.237.62:33459 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.2.237.2@KRBTEST.COM; defaulting to no policy
I20260620 23:00:18.591517 13322 ts_manager.cc:194] Registered new tserver with Master: 110aa0bbf1064e38adb996708256e7c8 (127.2.237.1:45781)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.2@KRBTEST.COM" created.
I20260620 23:00:18.593303 13322 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.1@KRBTEST.COM'} at 127.2.237.1:58285
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
I20260620 23:00:18.621742 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:0
--local_ip_for_outbound_sockets=127.2.237.2
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:33459
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:43369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:00:18.960016 13517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:00:18.960412 13517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:00:18.960603 13517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:00:18.968219 13517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:00:18.968416 13517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:00:18.968606 13517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 23:00:18.979014 13517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:43369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:33459
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:00:18.980535 13517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:00:18.981928 13517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:00:18.996493 13523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:00:18.996501 13522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:00:18.996417 13525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:00:18.997190 13517 server_base.cc:1034] running on GCE node
I20260620 23:00:18.998019 13517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:00:18.999179 13517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:00:19.000413 13517 hybrid_clock.cc:648] HybridClock initialized: now 1781996419000379 us; error 65 us; skew 500 ppm
I20260620 23:00:19.000777 13517 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:00:18 dist-test-slave-f63b krb5kdc[13277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996418, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:00:19.004057 13517 init.cc:377] Logged in from keytab as kudu/127.2.237.2@KRBTEST.COM (short username kudu)
I20260620 23:00:19.005602 13517 webserver.cc:460] Webserver started at http://127.2.237.2:45913/ using document root <none> and password file <none>
I20260620 23:00:19.006366 13517 fs_manager.cc:359] Metadata directory not provided
I20260620 23:00:19.006526 13517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:00:19.006870 13517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:00:19.010057 13517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/data/instance:
uuid: "336def00b2be4937967aa194ccf8c5d2"
format_stamp: "Formatted at 2026-06-20 23:00:19 on dist-test-slave-f63b"
server_key: "8c6fbda329c88e08b9594ea9306c865d"
server_key_iv: "75624462a6d0a753bfa44b844852a52e"
server_key_version: "encryptionkey@0"
I20260620 23:00:19.010950 13517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance:
uuid: "336def00b2be4937967aa194ccf8c5d2"
format_stamp: "Formatted at 2026-06-20 23:00:19 on dist-test-slave-f63b"
server_key: "8c6fbda329c88e08b9594ea9306c865d"
server_key_iv: "75624462a6d0a753bfa44b844852a52e"
server_key_version: "encryptionkey@0"
I20260620 23:00:19.016309 13517 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260620 23:00:19.019948 13532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:00:19.021312 13517 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 23:00:19.021512 13517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "336def00b2be4937967aa194ccf8c5d2"
format_stamp: "Formatted at 2026-06-20 23:00:19 on dist-test-slave-f63b"
server_key: "8c6fbda329c88e08b9594ea9306c865d"
server_key_iv: "75624462a6d0a753bfa44b844852a52e"
server_key_version: "encryptionkey@0"
I20260620 23:00:19.021726 13517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:00:19.051445 13517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:00:19.056192 13517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:00:19.056493 13517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:00:19.057507 13517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:00:19.058931 13517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:00:19.059011 13517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:00:19.059135 13517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:00:19.059190 13517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:00:19.092658 13517 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:35423
I20260620 23:00:19.092700 13644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:35423 every 8 connection(s)
I20260620 23:00:19.093979 13517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 23:00:19.096069 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 13517
I20260620 23:00:19.096282 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance
I20260620 23:00:19.096809 2996 external_mini_cluster.cc:1428] Setting key a645978903e2a422937364831a46ac77
Jun 20 23:00:19 dist-test-slave-f63b krb5kdc[13277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996418, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:00:19.116353 13647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:19.096762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:49957 (local address 127.2.237.62:33459)
0620 23:00:19.097823 (+ 1061us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:19.097829 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:19.098534 (+ 705us) server_negotiation.cc:407] Connection header received
0620 23:00:19.100391 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:19.100400 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:19.100497 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:19.100623 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:19.102151 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.103377 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.105051 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.105841 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.105984 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:19.110216 (+ 4232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:00:19.110221 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:00:19.110225 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:00:19.110260 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:00:19.113076 (+ 2816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:00:19.113865 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:00:19.113871 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:00:19.113875 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:00:19.114017 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:00:19.114465 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:00:19.114477 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:00:19.114480 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:00:19.114813 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:00:19.114973 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0620 23:00:19.115722 (+ 749us) server_negotiation.cc:299] Negotiation successful
0620 23:00:19.115961 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":940,"thread_start_us":832,"threads_started":1}
I20260620 23:00:19.117518 13648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:19.097307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:33459 (local address 127.2.237.2:49957)
0620 23:00:19.098363 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:19.098417 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:19.100108 (+ 1691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:19.100823 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:19.100832 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:19.101324 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:19.101966 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:19.101984 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.103547 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.103551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:19.104898 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:19.104910 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.106073 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.106083 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:19.106404 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:19.107231 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:19.107255 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:19.109924 (+ 2669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:00:19.113256 (+ 3332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:00:19.113266 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:00:19.113286 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:00:19.113734 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:00:19.114131 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:00:19.114136 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:00:19.114140 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:00:19.114344 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:00:19.114947 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:00:19.114955 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:00:19.115352 (+ 397us) client_negotiation.cc:769] Sending connection context
0620 23:00:19.115635 (+ 283us) client_negotiation.cc:240] Negotiation successful
0620 23:00:19.116238 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"thread_start_us":594,"threads_started":1}
I20260620 23:00:19.119046 13645 heartbeater.cc:347] Connected to a master server at 127.2.237.62:33459
I20260620 23:00:19.119364 13645 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.2.237.3@KRBTEST.COM; defaulting to no policy
I20260620 23:00:19.120160 13645 heartbeater.cc:511] Master 127.2.237.62:33459 requested a full tablet report, sending...
I20260620 23:00:19.121662 13322 ts_manager.cc:194] Registered new tserver with Master: 336def00b2be4937967aa194ccf8c5d2 (127.2.237.2:35423)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.3@KRBTEST.COM" created.
I20260620 23:00:19.122686 13322 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.2@KRBTEST.COM'} at 127.2.237.2:49957
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
I20260620 23:00:19.149883 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:0
--local_ip_for_outbound_sockets=127.2.237.3
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:33459
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:43369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:00:19.488852 13651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:00:19.489161 13651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:00:19.489259 13651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:00:19.496743 13651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:00:19.496876 13651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:00:19.496992 13651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 23:00:19.508745 13651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:43369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:33459
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:00:19.510054 13651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:00:19.511602 13651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:00:19.525175 13656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:00:19.525141 13659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:00:19.525206 13657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:00:19.526052 13651 server_base.cc:1034] running on GCE node
I20260620 23:00:19.526870 13651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:00:19.527910 13651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:00:19.529163 13651 hybrid_clock.cc:648] HybridClock initialized: now 1781996419529107 us; error 41 us; skew 500 ppm
I20260620 23:00:19.529615 13651 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:00:19 dist-test-slave-f63b krb5kdc[13277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996419, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:00:19.533217 13651 init.cc:377] Logged in from keytab as kudu/127.2.237.3@KRBTEST.COM (short username kudu)
I20260620 23:00:19.534895 13651 webserver.cc:460] Webserver started at http://127.2.237.3:40259/ using document root <none> and password file <none>
I20260620 23:00:19.535586 13651 fs_manager.cc:359] Metadata directory not provided
I20260620 23:00:19.535687 13651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:00:19.535949 13651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:00:19.539086 13651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/data/instance:
uuid: "0cc2473fb7414332b78979c6978b9388"
format_stamp: "Formatted at 2026-06-20 23:00:19 on dist-test-slave-f63b"
server_key: "d1704afefe99807ab2fbe92d45c309b4"
server_key_iv: "8684748d613b3b9dfbc1f50a2c1cf936"
server_key_version: "encryptionkey@0"
I20260620 23:00:19.539881 13651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance:
uuid: "0cc2473fb7414332b78979c6978b9388"
format_stamp: "Formatted at 2026-06-20 23:00:19 on dist-test-slave-f63b"
server_key: "d1704afefe99807ab2fbe92d45c309b4"
server_key_iv: "8684748d613b3b9dfbc1f50a2c1cf936"
server_key_version: "encryptionkey@0"
I20260620 23:00:19.545400 13651 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260620 23:00:19.549347 13666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:00:19.551002 13651 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260620 23:00:19.551193 13651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "0cc2473fb7414332b78979c6978b9388"
format_stamp: "Formatted at 2026-06-20 23:00:19 on dist-test-slave-f63b"
server_key: "d1704afefe99807ab2fbe92d45c309b4"
server_key_iv: "8684748d613b3b9dfbc1f50a2c1cf936"
server_key_version: "encryptionkey@0"
I20260620 23:00:19.551421 13651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:00:19.591921 13651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:00:19.596506 13651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:00:19.596634 13511 heartbeater.cc:503] Master 127.2.237.62:33459 was elected leader, sending a full tablet report...
I20260620 23:00:19.596895 13651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:00:19.597970 13651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:00:19.599452 13651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:00:19.599567 13651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:00:19.599721 13651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:00:19.599795 13651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:00:19.634362 13651 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:42995
I20260620 23:00:19.634411 13778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:42995 every 8 connection(s)
I20260620 23:00:19.635715 13651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 23:00:19.643322 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 13651
I20260620 23:00:19.643530 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance
I20260620 23:00:19.644114 2996 external_mini_cluster.cc:1428] Setting key fb5a60d4d4b3aa5098d1c3076fe9239e
Jun 20 23:00:19 dist-test-slave-f63b krb5kdc[13277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996419, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:00:19.657334 13781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:19.638218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:35233 (local address 127.2.237.62:33459)
0620 23:00:19.638722 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:19.638727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:19.639863 (+ 1136us) server_negotiation.cc:407] Connection header received
0620 23:00:19.641762 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:19.641769 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:19.641892 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:19.642022 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:19.644094 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.645097 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.647228 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.647698 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.647799 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:19.651632 (+ 3833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:00:19.651637 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:00:19.651640 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:00:19.651682 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:00:19.654186 (+ 2504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:00:19.655030 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:00:19.655034 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:00:19.655036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:00:19.655155 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:00:19.655679 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:00:19.655682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:00:19.655684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:00:19.656029 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:00:19.656144 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0620 23:00:19.656877 (+ 733us) server_negotiation.cc:299] Negotiation successful
0620 23:00:19.657054 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":248,"threads_started":1}
I20260620 23:00:19.658567 13782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:19.638812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:33459 (local address 127.2.237.3:35233)
0620 23:00:19.639660 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:19.639729 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:19.641468 (+ 1739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:19.642281 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:19.642290 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:19.642936 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:00:19.643836 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:19.643869 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.645255 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.645260 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:19.647087 (+ 1827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:19.647101 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.647916 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.647926 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:19.648212 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:19.648956 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:00:19.648980 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:00:19.651421 (+ 2441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:00:19.654332 (+ 2911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:00:19.654339 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:00:19.654353 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:00:19.654919 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:00:19.655271 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:00:19.655275 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:00:19.655278 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:00:19.655563 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:00:19.656141 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:00:19.656147 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:00:19.656544 (+ 397us) client_negotiation.cc:769] Sending connection context
0620 23:00:19.656796 (+ 252us) client_negotiation.cc:240] Negotiation successful
0620 23:00:19.657218 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":272,"threads_started":1}
I20260620 23:00:19.660120 13779 heartbeater.cc:347] Connected to a master server at 127.2.237.62:33459
I20260620 23:00:19.660419 13779 heartbeater.cc:464] Registering TS with master...
I20260620 23:00:19.661238 13779 heartbeater.cc:511] Master 127.2.237.62:33459 requested a full tablet report, sending...
I20260620 23:00:19.662658 13322 ts_manager.cc:194] Registered new tserver with Master: 0cc2473fb7414332b78979c6978b9388 (127.2.237.3:42995)
I20260620 23:00:19.663419 13322 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.3@KRBTEST.COM'} at 127.2.237.3:35233
I20260620 23:00:19.672634 2996 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:00:19.690050 13781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:19.677229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41720 (local address 127.2.237.62:33459)
0620 23:00:19.677400 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:19.677405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:19.677840 (+ 435us) server_negotiation.cc:407] Connection header received
0620 23:00:19.677978 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:19.677981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:19.678048 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:19.678162 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:19.679821 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.680667 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.682002 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.682412 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.682517 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:19.685024 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:00:19.685029 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:00:19.685033 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:00:19.685064 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:00:19.687777 (+ 2713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:00:19.688427 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:00:19.688430 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:00:19.688433 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:00:19.688564 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:00:19.688898 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:00:19.688901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:00:19.688903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:00:19.689199 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:00:19.689377 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0620 23:00:19.689662 (+ 285us) server_negotiation.cc:299] Negotiation successful
0620 23:00:19.689824 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 23:00:19.694238 13322 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41720:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:00:19.701241 13322 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:00:19.729415 13797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:19.715257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49578 (local address 127.2.237.3:42995)
0620 23:00:19.716128 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:19.716141 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:19.721120 (+ 4979us) server_negotiation.cc:407] Connection header received
0620 23:00:19.721375 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:19.721381 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:19.721556 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:19.721704 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:00:19.722971 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.724533 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.726791 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.728135 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.728277 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:19.728540 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0620 23:00:19.728854 (+ 314us) server_negotiation.cc:299] Negotiation successful
0620 23:00:19.729028 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":543,"threads_started":1}
I20260620 23:00:19.729617 13794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:19.714055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:42995 (local address 127.0.0.1:49578)
0620 23:00:19.721013 (+ 6958us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:19.721029 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:19.721211 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:19.722030 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:19.722034 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:19.722065 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:00:19.722693 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:19.722702 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.724649 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.724654 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:19.726632 (+ 1978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:19.726650 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.728346 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.728353 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:19.728566 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:19.728624 (+ 58us) client_negotiation.cc:769] Sending connection context
0620 23:00:19.728760 (+ 136us) client_negotiation.cc:240] Negotiation successful
0620 23:00:19.728965 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6832,"mutex_wait_us":24,"thread_start_us":6321,"threads_started":1}
I20260620 23:00:19.730487 13795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:19.713938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:45781 (local address 127.0.0.1:60100)
0620 23:00:19.720699 (+ 6761us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:19.720727 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:19.720997 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:19.721815 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:19.721822 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:19.721852 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:00:19.722690 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:19.722702 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.724128 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.724132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:19.727557 (+ 3425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:19.727567 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.729523 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.729530 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:19.729746 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:19.729782 (+ 36us) client_negotiation.cc:769] Sending connection context
0620 23:00:19.729936 (+ 154us) client_negotiation.cc:240] Negotiation successful
0620 23:00:19.730071 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6609,"mutex_wait_us":151,"spinlock_wait_cycles":1920,"thread_start_us":6811,"threads_started":1}
I20260620 23:00:19.730684 13793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:19.713719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58750 (local address 127.2.237.2:35423)
0620 23:00:19.720306 (+ 6587us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:19.720318 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:19.720362 (+ 44us) server_negotiation.cc:407] Connection header received
0620 23:00:19.721181 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:19.721193 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:19.721497 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:19.721708 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:00:19.722813 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.724441 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.727315 (+ 2874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.728764 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.728885 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:19.729070 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0620 23:00:19.730182 (+ 1112us) server_negotiation.cc:299] Negotiation successful
0620 23:00:19.730344 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6374,"thread_start_us":6200,"threads_started":1}
I20260620 23:00:19.731240 13792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:19.713494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:35423 (local address 127.0.0.1:58750)
0620 23:00:19.714303 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:19.714343 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:19.714518 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:19.721957 (+ 7439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:19.721962 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:19.721996 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:00:19.722554 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:19.722626 (+ 72us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.724604 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.724611 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:19.727136 (+ 2525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:19.727156 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.729220 (+ 2064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.729233 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:19.729447 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:19.729487 (+ 40us) client_negotiation.cc:769] Sending connection context
0620 23:00:19.729577 (+ 90us) client_negotiation.cc:240] Negotiation successful
0620 23:00:19.729711 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"mutex_wait_us":181,"thread_start_us":601,"threads_started":1}
I20260620 23:00:19.731563 13796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:19.714774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60100 (local address 127.2.237.1:45781)
0620 23:00:19.715527 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:19.715541 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:19.720868 (+ 5327us) server_negotiation.cc:407] Connection header received
0620 23:00:19.721213 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:19.721221 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:19.721503 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:19.721708 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:00:19.722933 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.723982 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.727707 (+ 3725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.729012 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.729137 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:19.729364 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0620 23:00:19.731034 (+ 1670us) server_negotiation.cc:299] Negotiation successful
0620 23:00:19.731217 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":814,"threads_started":1}
I20260620 23:00:19.733734 13714 tablet_service.cc:1467] Processing CreateTablet for tablet 5fb8d2f70af84b27a747d9e907fbda58 (DEFAULT_TABLE table=test-table [id=45c7dec9b32245a2a77bd309f786102c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:00:19.735059 13714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fb8d2f70af84b27a747d9e907fbda58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:00:19.734783 13580 tablet_service.cc:1467] Processing CreateTablet for tablet 5fb8d2f70af84b27a747d9e907fbda58 (DEFAULT_TABLE table=test-table [id=45c7dec9b32245a2a77bd309f786102c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:00:19.735095 13446 tablet_service.cc:1467] Processing CreateTablet for tablet 5fb8d2f70af84b27a747d9e907fbda58 (DEFAULT_TABLE table=test-table [id=45c7dec9b32245a2a77bd309f786102c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:00:19.736140 13580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fb8d2f70af84b27a747d9e907fbda58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:00:19.737646 13446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fb8d2f70af84b27a747d9e907fbda58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:00:19.745954 13798 tablet_bootstrap.cc:492] T 5fb8d2f70af84b27a747d9e907fbda58 P 0cc2473fb7414332b78979c6978b9388: Bootstrap starting.
I20260620 23:00:19.749217 13798 tablet_bootstrap.cc:654] T 5fb8d2f70af84b27a747d9e907fbda58 P 0cc2473fb7414332b78979c6978b9388: Neither blocks nor log segments found. Creating new log.
I20260620 23:00:19.750433 13798 log.cc:824] T 5fb8d2f70af84b27a747d9e907fbda58 P 0cc2473fb7414332b78979c6978b9388: Log is configured to *not* fsync() on all Append() calls
I20260620 23:00:19.750751 13799 tablet_bootstrap.cc:492] T 5fb8d2f70af84b27a747d9e907fbda58 P 336def00b2be4937967aa194ccf8c5d2: Bootstrap starting.
I20260620 23:00:19.755498 13799 tablet_bootstrap.cc:654] T 5fb8d2f70af84b27a747d9e907fbda58 P 336def00b2be4937967aa194ccf8c5d2: Neither blocks nor log segments found. Creating new log.
I20260620 23:00:19.757157 13799 log.cc:824] T 5fb8d2f70af84b27a747d9e907fbda58 P 336def00b2be4937967aa194ccf8c5d2: Log is configured to *not* fsync() on all Append() calls
I20260620 23:00:19.759075 13801 tablet_bootstrap.cc:492] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8: Bootstrap starting.
I20260620 23:00:19.760629 13798 tablet_bootstrap.cc:492] T 5fb8d2f70af84b27a747d9e907fbda58 P 0cc2473fb7414332b78979c6978b9388: No bootstrap required, opened a new log
I20260620 23:00:19.760673 13799 tablet_bootstrap.cc:492] T 5fb8d2f70af84b27a747d9e907fbda58 P 336def00b2be4937967aa194ccf8c5d2: No bootstrap required, opened a new log
I20260620 23:00:19.760946 13799 ts_tablet_manager.cc:1397] T 5fb8d2f70af84b27a747d9e907fbda58 P 336def00b2be4937967aa194ccf8c5d2: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260620 23:00:19.760972 13798 ts_tablet_manager.cc:1397] T 5fb8d2f70af84b27a747d9e907fbda58 P 0cc2473fb7414332b78979c6978b9388: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.004s
I20260620 23:00:19.762790 13801 tablet_bootstrap.cc:654] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8: Neither blocks nor log segments found. Creating new log.
I20260620 23:00:19.764071 13801 log.cc:824] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8: Log is configured to *not* fsync() on all Append() calls
I20260620 23:00:19.767325 13801 tablet_bootstrap.cc:492] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8: No bootstrap required, opened a new log
I20260620 23:00:19.767623 13801 ts_tablet_manager.cc:1397] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.006s
I20260620 23:00:19.769341 13799 raft_consensus.cc:348] T 5fb8d2f70af84b27a747d9e907fbda58 P 336def00b2be4937967aa194ccf8c5d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "336def00b2be4937967aa194ccf8c5d2" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 35423 } } peers { permanent_uuid: "110aa0bbf1064e38adb996708256e7c8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 45781 } } peers { permanent_uuid: "0cc2473fb7414332b78979c6978b9388" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 42995 } }
I20260620 23:00:19.769737 13799 raft_consensus.cc:374] T 5fb8d2f70af84b27a747d9e907fbda58 P 336def00b2be4937967aa194ccf8c5d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:00:19.769834 13799 raft_consensus.cc:729] T 5fb8d2f70af84b27a747d9e907fbda58 P 336def00b2be4937967aa194ccf8c5d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 336def00b2be4937967aa194ccf8c5d2, State: Initialized, Role: FOLLOWER
I20260620 23:00:19.770471 13799 consensus_queue.cc:260] T 5fb8d2f70af84b27a747d9e907fbda58 P 336def00b2be4937967aa194ccf8c5d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "336def00b2be4937967aa194ccf8c5d2" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 35423 } } peers { permanent_uuid: "110aa0bbf1064e38adb996708256e7c8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 45781 } } peers { permanent_uuid: "0cc2473fb7414332b78979c6978b9388" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 42995 } }
I20260620 23:00:19.771760 13645 heartbeater.cc:503] Master 127.2.237.62:33459 was elected leader, sending a full tablet report...
I20260620 23:00:19.771540 13798 raft_consensus.cc:348] T 5fb8d2f70af84b27a747d9e907fbda58 P 0cc2473fb7414332b78979c6978b9388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "336def00b2be4937967aa194ccf8c5d2" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 35423 } } peers { permanent_uuid: "110aa0bbf1064e38adb996708256e7c8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 45781 } } peers { permanent_uuid: "0cc2473fb7414332b78979c6978b9388" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 42995 } }
I20260620 23:00:19.771981 13798 raft_consensus.cc:374] T 5fb8d2f70af84b27a747d9e907fbda58 P 0cc2473fb7414332b78979c6978b9388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:00:19.772071 13798 raft_consensus.cc:729] T 5fb8d2f70af84b27a747d9e907fbda58 P 0cc2473fb7414332b78979c6978b9388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cc2473fb7414332b78979c6978b9388, State: Initialized, Role: FOLLOWER
I20260620 23:00:19.772478 13799 ts_tablet_manager.cc:1428] T 5fb8d2f70af84b27a747d9e907fbda58 P 336def00b2be4937967aa194ccf8c5d2: Time spent starting tablet: real 0.011s user 0.011s sys 0.001s
I20260620 23:00:19.772680 13798 consensus_queue.cc:260] T 5fb8d2f70af84b27a747d9e907fbda58 P 0cc2473fb7414332b78979c6978b9388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "336def00b2be4937967aa194ccf8c5d2" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 35423 } } peers { permanent_uuid: "110aa0bbf1064e38adb996708256e7c8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 45781 } } peers { permanent_uuid: "0cc2473fb7414332b78979c6978b9388" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 42995 } }
I20260620 23:00:19.779314 13779 heartbeater.cc:503] Master 127.2.237.62:33459 was elected leader, sending a full tablet report...
I20260620 23:00:19.780275 13798 ts_tablet_manager.cc:1428] T 5fb8d2f70af84b27a747d9e907fbda58 P 0cc2473fb7414332b78979c6978b9388: Time spent starting tablet: real 0.019s user 0.020s sys 0.000s
I20260620 23:00:19.780346 13801 raft_consensus.cc:348] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "336def00b2be4937967aa194ccf8c5d2" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 35423 } } peers { permanent_uuid: "110aa0bbf1064e38adb996708256e7c8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 45781 } } peers { permanent_uuid: "0cc2473fb7414332b78979c6978b9388" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 42995 } }
I20260620 23:00:19.780884 13801 raft_consensus.cc:374] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:00:19.781002 13801 raft_consensus.cc:729] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 110aa0bbf1064e38adb996708256e7c8, State: Initialized, Role: FOLLOWER
I20260620 23:00:19.781698 13801 consensus_queue.cc:260] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "336def00b2be4937967aa194ccf8c5d2" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 35423 } } peers { permanent_uuid: "110aa0bbf1064e38adb996708256e7c8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 45781 } } peers { permanent_uuid: "0cc2473fb7414332b78979c6978b9388" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 42995 } }
I20260620 23:00:19.783450 13801 ts_tablet_manager.cc:1428] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
W20260620 23:00:19.813222 13512 tablet.cc:2369] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:00:19.846141 13646 tablet.cc:2369] T 5fb8d2f70af84b27a747d9e907fbda58 P 336def00b2be4937967aa194ccf8c5d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:00:19.883409 13806 raft_consensus.cc:482] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:00:19.883724 13806 raft_consensus.cc:504] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "336def00b2be4937967aa194ccf8c5d2" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 35423 } } peers { permanent_uuid: "110aa0bbf1064e38adb996708256e7c8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 45781 } } peers { permanent_uuid: "0cc2473fb7414332b78979c6978b9388" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 42995 } }
I20260620 23:00:19.885820 13806 leader_election.cc:288] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 336def00b2be4937967aa194ccf8c5d2 (127.2.237.2:35423), 0cc2473fb7414332b78979c6978b9388 (127.2.237.3:42995)
W20260620 23:00:19.887405 13780 tablet.cc:2369] T 5fb8d2f70af84b27a747d9e907fbda58 P 0cc2473fb7414332b78979c6978b9388: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:00:19.894608 13797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:19.886418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:47445 (local address 127.2.237.3:42995)
0620 23:00:19.886667 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:19.886675 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:19.887258 (+ 583us) server_negotiation.cc:407] Connection header received
0620 23:00:19.887476 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:19.887482 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:19.887562 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:19.887653 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:00:19.888406 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.889406 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.892025 (+ 2619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.893277 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.893419 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:19.893520 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0620 23:00:19.894176 (+ 656us) server_negotiation.cc:299] Negotiation successful
0620 23:00:19.894364 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260620 23:00:19.894586 13793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:19.886168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:57035 (local address 127.2.237.2:35423)
0620 23:00:19.886514 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:19.886524 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:19.887027 (+ 503us) server_negotiation.cc:407] Connection header received
0620 23:00:19.887264 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:19.887270 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:19.887348 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:19.887458 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:00:19.888073 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.889136 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.892032 (+ 2896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.893277 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.893417 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:19.893519 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0620 23:00:19.894176 (+ 657us) server_negotiation.cc:299] Negotiation successful
0620 23:00:19.894317 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260620 23:00:19.894795 13808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:19.886129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:42995 (local address 127.2.237.1:47445)
0620 23:00:19.887173 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:19.887189 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:19.887335 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:19.887745 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:19.887751 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:19.887784 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:00:19.888227 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:19.888242 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.889541 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.889547 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:19.891851 (+ 2304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:19.891867 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.893573 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.893581 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:19.893876 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:19.893972 (+ 96us) client_negotiation.cc:769] Sending connection context
0620 23:00:19.894076 (+ 104us) client_negotiation.cc:240] Negotiation successful
0620 23:00:19.894277 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":925,"mutex_wait_us":165,"thread_start_us":752,"threads_started":1}
I20260620 23:00:19.895169 13807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:19.886064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:35423 (local address 127.2.237.1:57035)
0620 23:00:19.886862 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0620 23:00:19.886904 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 23:00:19.887096 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:00:19.887501 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:00:19.887505 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:00:19.887525 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:00:19.887916 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:19.887927 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.889325 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.889329 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:19.891853 (+ 2524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:00:19.891867 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.893591 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.893597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:00:19.893878 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:19.893973 (+ 95us) client_negotiation.cc:769] Sending connection context
0620 23:00:19.894076 (+ 103us) client_negotiation.cc:240] Negotiation successful
0620 23:00:19.894321 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"mutex_wait_us":169,"thread_start_us":488,"threads_started":1}
I20260620 23:00:19.895947 13734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fb8d2f70af84b27a747d9e907fbda58" candidate_uuid: "110aa0bbf1064e38adb996708256e7c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cc2473fb7414332b78979c6978b9388" is_pre_election: true
I20260620 23:00:19.896128 13600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fb8d2f70af84b27a747d9e907fbda58" candidate_uuid: "110aa0bbf1064e38adb996708256e7c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "336def00b2be4937967aa194ccf8c5d2" is_pre_election: true
I20260620 23:00:19.896576 13734 raft_consensus.cc:2454] T 5fb8d2f70af84b27a747d9e907fbda58 P 0cc2473fb7414332b78979c6978b9388 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 110aa0bbf1064e38adb996708256e7c8 in term 0.
I20260620 23:00:19.896600 13600 raft_consensus.cc:2454] T 5fb8d2f70af84b27a747d9e907fbda58 P 336def00b2be4937967aa194ccf8c5d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 110aa0bbf1064e38adb996708256e7c8 in term 0.
I20260620 23:00:19.897320 13399 leader_election.cc:302] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0cc2473fb7414332b78979c6978b9388, 110aa0bbf1064e38adb996708256e7c8, 336def00b2be4937967aa194ccf8c5d2; no voters:
I20260620 23:00:19.897760 13806 raft_consensus.cc:2781] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:00:19.897920 13806 raft_consensus.cc:482] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:00:19.898028 13806 raft_consensus.cc:3037] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:00:19.900482 13806 raft_consensus.cc:504] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "336def00b2be4937967aa194ccf8c5d2" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 35423 } } peers { permanent_uuid: "110aa0bbf1064e38adb996708256e7c8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 45781 } } peers { permanent_uuid: "0cc2473fb7414332b78979c6978b9388" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 42995 } }
I20260620 23:00:19.901108 13806 leader_election.cc:288] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8 [CANDIDATE]: Term 1 election: Requested vote from peers 336def00b2be4937967aa194ccf8c5d2 (127.2.237.2:35423), 0cc2473fb7414332b78979c6978b9388 (127.2.237.3:42995)
I20260620 23:00:19.901549 13600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fb8d2f70af84b27a747d9e907fbda58" candidate_uuid: "110aa0bbf1064e38adb996708256e7c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "336def00b2be4937967aa194ccf8c5d2"
I20260620 23:00:19.901615 13734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fb8d2f70af84b27a747d9e907fbda58" candidate_uuid: "110aa0bbf1064e38adb996708256e7c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cc2473fb7414332b78979c6978b9388"
I20260620 23:00:19.901791 13734 raft_consensus.cc:3037] T 5fb8d2f70af84b27a747d9e907fbda58 P 0cc2473fb7414332b78979c6978b9388 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:00:19.901777 13600 raft_consensus.cc:3037] T 5fb8d2f70af84b27a747d9e907fbda58 P 336def00b2be4937967aa194ccf8c5d2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:00:19.905000 13600 raft_consensus.cc:2454] T 5fb8d2f70af84b27a747d9e907fbda58 P 336def00b2be4937967aa194ccf8c5d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 110aa0bbf1064e38adb996708256e7c8 in term 1.
I20260620 23:00:19.905000 13734 raft_consensus.cc:2454] T 5fb8d2f70af84b27a747d9e907fbda58 P 0cc2473fb7414332b78979c6978b9388 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 110aa0bbf1064e38adb996708256e7c8 in term 1.
I20260620 23:00:19.905483 13400 leader_election.cc:302] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 110aa0bbf1064e38adb996708256e7c8, 336def00b2be4937967aa194ccf8c5d2; no voters:
I20260620 23:00:19.905866 13806 raft_consensus.cc:2781] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:00:19.906450 13806 raft_consensus.cc:686] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8 [term 1 LEADER]: Becoming Leader. State: Replica: 110aa0bbf1064e38adb996708256e7c8, State: Running, Role: LEADER
I20260620 23:00:19.907056 13806 consensus_queue.cc:237] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "336def00b2be4937967aa194ccf8c5d2" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 35423 } } peers { permanent_uuid: "110aa0bbf1064e38adb996708256e7c8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 45781 } } peers { permanent_uuid: "0cc2473fb7414332b78979c6978b9388" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 42995 } }
I20260620 23:00:19.912001 13322 catalog_manager.cc:5495] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 110aa0bbf1064e38adb996708256e7c8 (127.2.237.1). New cstate: current_term: 1 leader_uuid: "110aa0bbf1064e38adb996708256e7c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "336def00b2be4937967aa194ccf8c5d2" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 35423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "110aa0bbf1064e38adb996708256e7c8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 45781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0cc2473fb7414332b78979c6978b9388" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 42995 } health_report { overall_health: UNKNOWN } } }
I20260620 23:00:19.964922 13796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:19.959522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60104 (local address 127.2.237.1:45781)
0620 23:00:19.959752 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:19.959759 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:19.959779 (+ 20us) server_negotiation.cc:407] Connection header received
0620 23:00:19.959972 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:19.959976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:19.960068 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:19.960240 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:00:19.960660 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.961367 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.962768 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:19.963182 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:19.963315 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:19.963809 (+ 494us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:00:19.964351 (+ 542us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:00:19.964521 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0620 23:00:19.964570 (+ 49us) server_negotiation.cc:299] Negotiation successful
0620 23:00:19.964650 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260620 23:00:19.973953 13600 raft_consensus.cc:1261] T 5fb8d2f70af84b27a747d9e907fbda58 P 336def00b2be4937967aa194ccf8c5d2 [term 1 FOLLOWER]: Refusing update from remote peer 110aa0bbf1064e38adb996708256e7c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:00:19.973953 13734 raft_consensus.cc:1261] T 5fb8d2f70af84b27a747d9e907fbda58 P 0cc2473fb7414332b78979c6978b9388 [term 1 FOLLOWER]: Refusing update from remote peer 110aa0bbf1064e38adb996708256e7c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:00:19.975282 13809 consensus_queue.cc:1035] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cc2473fb7414332b78979c6978b9388" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:00:19.975687 13806 consensus_queue.cc:1035] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "336def00b2be4937967aa194ccf8c5d2" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 35423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:00:19.987929 13812 mvcc.cc:205] Tried to move back new op lower bound from 7299057336201781248 to 7299057335943364608. Current Snapshot: MvccSnapshot[applied={T|T < 7299057336201781248}]
I20260620 23:00:20.011960 13322 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:00:20.012202 13322 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:00:20.016464 13322 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b76afa314106411492df21aaf6cdf7a7: Sending DeleteTablet for 3 replicas of tablet 5fb8d2f70af84b27a747d9e907fbda58
I20260620 23:00:20.018435 13580 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fb8d2f70af84b27a747d9e907fbda58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:00:20 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:58750
I20260620 23:00:20.018694 13446 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fb8d2f70af84b27a747d9e907fbda58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:00:20 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:60100
I20260620 23:00:20.019723 13714 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fb8d2f70af84b27a747d9e907fbda58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:00:20 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:49578
I20260620 23:00:20.028853 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 13383
I20260620 23:00:20.030887 13822 tablet_replica.cc:330] T 5fb8d2f70af84b27a747d9e907fbda58 P 336def00b2be4937967aa194ccf8c5d2: stopping tablet replica
I20260620 23:00:20.031414 13822 raft_consensus.cc:2229] T 5fb8d2f70af84b27a747d9e907fbda58 P 336def00b2be4937967aa194ccf8c5d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:00:20.031982 13822 raft_consensus.cc:2258] T 5fb8d2f70af84b27a747d9e907fbda58 P 336def00b2be4937967aa194ccf8c5d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:00:20.034323 13823 tablet_replica.cc:330] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8: stopping tablet replica
I20260620 23:00:20.034837 13823 raft_consensus.cc:2229] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8 [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:00:20.035979 13824 tablet_replica.cc:330] T 5fb8d2f70af84b27a747d9e907fbda58 P 0cc2473fb7414332b78979c6978b9388: stopping tablet replica
I20260620 23:00:20.036466 13824 raft_consensus.cc:2229] T 5fb8d2f70af84b27a747d9e907fbda58 P 0cc2473fb7414332b78979c6978b9388 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:00:20.037031 13824 raft_consensus.cc:2258] T 5fb8d2f70af84b27a747d9e907fbda58 P 0cc2473fb7414332b78979c6978b9388 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:00:20.039665 13822 ts_tablet_manager.cc:1905] T 5fb8d2f70af84b27a747d9e907fbda58 P 336def00b2be4937967aa194ccf8c5d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:00:20.042857 13823 raft_consensus.cc:2258] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:00:20.045607 13824 ts_tablet_manager.cc:1905] T 5fb8d2f70af84b27a747d9e907fbda58 P 0cc2473fb7414332b78979c6978b9388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:00:20.047421 13822 ts_tablet_manager.cc:1918] T 5fb8d2f70af84b27a747d9e907fbda58 P 336def00b2be4937967aa194ccf8c5d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:00:20.047600 13822 log.cc:1196] T 5fb8d2f70af84b27a747d9e907fbda58 P 336def00b2be4937967aa194ccf8c5d2: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-1/wal/wals/5fb8d2f70af84b27a747d9e907fbda58
I20260620 23:00:20.047756 13823 ts_tablet_manager.cc:1905] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:00:20.048166 13822 ts_tablet_manager.cc:1939] T 5fb8d2f70af84b27a747d9e907fbda58 P 336def00b2be4937967aa194ccf8c5d2: Deleting consensus metadata
I20260620 23:00:20.050343 13307 catalog_manager.cc:4841] TS 336def00b2be4937967aa194ccf8c5d2 (127.2.237.2:35423): tablet 5fb8d2f70af84b27a747d9e907fbda58 (table test-table [id=45c7dec9b32245a2a77bd309f786102c]) successfully deleted
I20260620 23:00:20.052557 13824 ts_tablet_manager.cc:1918] T 5fb8d2f70af84b27a747d9e907fbda58 P 0cc2473fb7414332b78979c6978b9388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:00:20.052735 13824 log.cc:1196] T 5fb8d2f70af84b27a747d9e907fbda58 P 0cc2473fb7414332b78979c6978b9388: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-2/wal/wals/5fb8d2f70af84b27a747d9e907fbda58
I20260620 23:00:20.053233 13824 ts_tablet_manager.cc:1939] T 5fb8d2f70af84b27a747d9e907fbda58 P 0cc2473fb7414332b78979c6978b9388: Deleting consensus metadata
Jun 20 23:00:20 dist-test-slave-f63b krb5kdc[13277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.1@KRBTEST.COM
I20260620 23:00:20.055114 13823 ts_tablet_manager.cc:1918] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:00:20.055230 13306 catalog_manager.cc:4841] TS 0cc2473fb7414332b78979c6978b9388 (127.2.237.3:42995): tablet 5fb8d2f70af84b27a747d9e907fbda58 (table test-table [id=45c7dec9b32245a2a77bd309f786102c]) successfully deleted
I20260620 23:00:20.055301 13823 log.cc:1196] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781996318807807-2996-0/minicluster-data/ts-0/wal/wals/5fb8d2f70af84b27a747d9e907fbda58
I20260620 23:00:20.055930 13823 ts_tablet_manager.cc:1939] T 5fb8d2f70af84b27a747d9e907fbda58 P 110aa0bbf1064e38adb996708256e7c8: Deleting consensus metadata
I20260620 23:00:20.057950 13309 catalog_manager.cc:4841] TS 110aa0bbf1064e38adb996708256e7c8 (127.2.237.1:45781): tablet 5fb8d2f70af84b27a747d9e907fbda58 (table test-table [id=45c7dec9b32245a2a77bd309f786102c]) successfully deleted
I20260620 23:00:20.066533 13796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:20.033017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60112 (local address 127.2.237.1:45781)
0620 23:00:20.033263 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:20.033273 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:20.043167 (+ 9894us) server_negotiation.cc:407] Connection header received
0620 23:00:20.043530 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:20.043542 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:20.043638 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:20.043753 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:20.045551 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:20.046880 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:20.048933 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:20.049643 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:20.049755 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:20.055244 (+ 5489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:00:20.055262 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:00:20.055283 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:00:20.055359 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:00:20.059650 (+ 4291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:00:20.060722 (+ 1072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:00:20.060732 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:00:20.060735 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:00:20.060970 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:00:20.061378 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:00:20.061384 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:00:20.061389 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:00:20.061986 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:00:20.062322 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0620 23:00:20.066064 (+ 3742us) server_negotiation.cc:299] Negotiation successful
0620 23:00:20.066272 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260620 23:00:20.067435 13506 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:00:20.189802 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 13383
I20260620 23:00:20.208638 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 13517
Jun 20 23:00:20 dist-test-slave-f63b krb5kdc[13277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.2@KRBTEST.COM
I20260620 23:00:20.225101 13793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:20.209517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58754 (local address 127.2.237.2:35423)
0620 23:00:20.209713 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:20.209717 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:20.209737 (+ 20us) server_negotiation.cc:407] Connection header received
0620 23:00:20.209834 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:20.209840 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:20.209905 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:20.209996 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:20.211283 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:20.212063 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:20.213367 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:20.213936 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:20.214062 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:20.219420 (+ 5358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:00:20.219436 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:00:20.219454 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:00:20.219496 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:00:20.222116 (+ 2620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:00:20.223090 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:00:20.223098 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:00:20.223100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:00:20.223227 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:00:20.223579 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:00:20.223582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:00:20.223584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:00:20.223958 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:00:20.224172 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0620 23:00:20.224465 (+ 293us) server_negotiation.cc:299] Negotiation successful
0620 23:00:20.224733 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260620 23:00:20.225876 13640 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:00:20.316855 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 13517
I20260620 23:00:20.334019 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 13651
Jun 20 23:00:20 dist-test-slave-f63b krb5kdc[13277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.3@KRBTEST.COM
I20260620 23:00:20.349764 13797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:00:20.334898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49586 (local address 127.2.237.3:42995)
0620 23:00:20.335089 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0620 23:00:20.335097 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:00:20.335184 (+ 87us) server_negotiation.cc:407] Connection header received
0620 23:00:20.335390 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:00:20.335394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:00:20.335475 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:00:20.335586 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:00:20.336838 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:20.337566 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:20.338975 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:00:20.339348 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:00:20.339484 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:00:20.343981 (+ 4497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:00:20.343997 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:00:20.344014 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:00:20.344049 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:00:20.347024 (+ 2975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:00:20.347710 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:00:20.347717 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:00:20.347719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:00:20.347854 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:00:20.348289 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:00:20.348293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:00:20.348295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:00:20.348674 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:00:20.348914 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0620 23:00:20.349170 (+ 256us) server_negotiation.cc:299] Negotiation successful
0620 23:00:20.349431 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260620 23:00:20.350427 13774 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:00:20.441931 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 13651
I20260620 23:00:20.459327 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 13290
I20260620 23:00:20.460456 13352 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:00:20.572243 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 13290
2026-06-20T23:00:20Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4433 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:00:20 dist-test-slave-f63b krb5kdc[13832](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:00:20 dist-test-slave-f63b krb5kdc[13832](info): set up 2 sockets
Jun 20 23:00:20 dist-test-slave-f63b krb5kdc[13832](info): commencing operation
krb5kdc: starting...
W20260620 23:00:21.691635 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 23:00:21 dist-test-slave-f63b krb5kdc[13832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:00:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:00:21Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.2.237.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.2.237.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.2.237.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/rangeradmin_127.2.237.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.2.237.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/rangeradmin_127.2.237.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.2.237.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.2.237.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.2.237.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/rangerlookup_127.2.237.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.2.237.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/rangerlookup_127.2.237.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.237.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.2.237.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.237.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/HTTP_127.2.237.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.237.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/HTTP_127.2.237.20@KRBTEST.COM.keytab.
I20260620 23:00:22.030130 2996 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskWPCN6h/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/postgres -l logfile start
2026-06-20 23:00:24.650 UTC [13862] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-20 23:00:24.650 UTC [13862] LOG: listening on IPv4 address "127.2.237.20", port 48037
2026-06-20 23:00:24.654 UTC [13862] LOG: listening on Unix socket "/tmp/.s.PGSQL.48037"
2026-06-20 23:00:24.679 UTC [13865] LOG: database system was shut down at 2026-06-20 23:00:23 UTC
2026-06-20 23:00:24.682 UTC [13862] LOG: database system is ready to accept connections
I20260620 23:00:25.667084 2996 mini_postgres.cc:96] Postgres bound to 48037
2026-06-20 23:00:25.685 UTC [13873] FATAL: database "slave" does not exist
127.2.237.20:48037 - accepting connections
I20260620 23:00:25.686110 2996 mini_ranger.cc:161] Starting Ranger...
I20260620 23:00:25.713985 2996 mini_ranger.cc:84] Created miniranger Postgres user
I20260620 23:00:26.205101 2996 mini_ranger.cc:87] Created ranger Postgres database
I20260620 23:00:26.205241 2996 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ranger-admin
2026-06-20 23:00:26,497 [I] DB FLAVOR :POSTGRES
2026-06-20 23:00:26,497 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 23:00:26,497 [I] Checking connection..
2026-06-20 23:00:26,497 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:48037/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-20 23:00:26,776 [I] Checking connection passed.
2026-06-20 23:00:26,777 [I] --------- Verifying version history table ---------
2026-06-20 23:00:26,777 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:48037/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 23:00:27,034 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 23:00:27,035 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:48037/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 23:00:27,308 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 23:00:27,308 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-20 23:00:27,308 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:48037/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-20 23:00:27.581 UTC [13937] WARNING: there is no transaction in progress
2026-06-20 23:00:27,587 [I] create_dbversion_catalog.sql file imported successfully
2026-06-20 23:00:27,588 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:48037/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 23:00:27,835 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-20 23:00:27,836 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 23:00:27,836 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:48037/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-20 23:00:28,081 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:48037/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-20 23:00:28,339 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:48037/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-f63b.c.gcp-upstream.internal','N') ;"
2026-06-20 23:00:28,574 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:48037/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 23:00:28,825 [I] Table x_portal_user does not exist in database ranger
2026-06-20 23:00:28,825 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:48037/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 23:00:29,075 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-20 23:00:29,075 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:48037/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 23:00:29,327 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:48037/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 23:00:29,597 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-20 23:00:29,598 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:48037/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-20 23:00:30.579 UTC [14045] WARNING: there is no transaction in progress
2026-06-20 23:00:30.595 UTC [14045] WARNING: there is no transaction in progress
2026-06-20 23:00:30.613 UTC [14045] WARNING: there is no transaction in progress
2026-06-20 23:00:30.631 UTC [14045] WARNING: there is no transaction in progress
2026-06-20 23:00:30.647 UTC [14045] WARNING: there is no transaction in progress
2026-06-20 23:00:30.667 UTC [14045] WARNING: there is no transaction in progress
2026-06-20 23:00:30.735 UTC [14045] WARNING: there is no transaction in progress
2026-06-20 23:00:30.742 UTC [14045] WARNING: there is no transaction in progress
2026-06-20 23:00:30.757 UTC [14045] WARNING: there is no transaction in progress
2026-06-20 23:00:30.768 UTC [14045] WARNING: there is no transaction in progress
2026-06-20 23:00:30.780 UTC [14045] WARNING: there is no transaction in progress
2026-06-20 23:00:30.795 UTC [14045] WARNING: there is no transaction in progress
2026-06-20 23:00:30.806 UTC [14045] WARNING: there is no transaction in progress
2026-06-20 23:00:30.814 UTC [14045] WARNING: there is no transaction in progress
2026-06-20 23:00:30.823 UTC [14045] WARNING: there is no transaction in progress
2026-06-20 23:00:30.830 UTC [14045] WARNING: there is no transaction in progress
2026-06-20 23:00:31,525 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-20 23:00:31,525 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:48037/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-20 23:00:31,775 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-20 23:00:31,775 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:48037/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 23:00:32,028 [I] Table x_portal_user already exists in database 'ranger'
2026-06-20 23:00:32,029 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:48037/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 23:00:32,285 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-20 23:00:32,286 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:48037/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 23:00:32,544 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:48037/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 23:00:32,805 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:48037/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-f63b.c.gcp-upstream.internal';"
2026-06-20 23:00:33,043 [I] CORE_DB_SCHEMA import status has been updated
2026-06-20 23:00:33,044 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:48037/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-20 23:00:33,288 [I] DB_PATCHES have already been applied
I20260620 23:00:33.294917 2996 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ranger-admin:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskWPCN6h/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260620 23:00:33.295043 2996 mini_ranger.cc:193] Using host: 127.2.237.20
I20260620 23:00:33.310948 2996 mini_ranger.cc:227] Ranger admin URL: http://127.2.237.20:58369
Jun 20, 2026 11:00:33 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 11:00:33 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 11:00:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ranger-admin/ews/webapp
Jun 20, 2026 11:00:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 20, 2026 11:00:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 20, 2026 11:00:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ranger-admin/ews/webapp].
Jun 20, 2026 11:00:34 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260620 23:00:34.452942 2996 mini_ranger.cc:160] Time spent starting Ranger: real 8.767s user 0.002s sys 0.042s
Jun 20, 2026 11:00:56 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 20, 2026 11:00:56 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 20, 2026 11:00:56 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 20, 2026 11:00:56 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 20, 2026 11:00:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 20, 2026 11:00:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 20, 2026 11:00:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 20, 2026 11:00:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 20, 2026 11:00:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 20, 2026 11:00:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 20, 2026 11:00:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 20, 2026 11:00:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 20, 2026 11:00:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 20, 2026 11:00:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 20, 2026 11:00:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 20, 2026 11:00:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 20, 2026 11:00:56 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 20, 2026 11:00:57 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260620 23:00:58.088968 2996 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.2.237.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.2.237.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.2.237.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/rangerkms_127.2.237.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.2.237.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/rangerkms_127.2.237.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.237.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.2.237.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.237.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/HTTP_127.2.237.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.237.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/HTTP_127.2.237.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 20 23:00:58 dist-test-slave-f63b krb5kdc[13832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996458, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260620 23:00:58.260560 2996 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260620 23:00:58.283449 2996 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260620 23:00:59.177753 2996 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260620 23:00:59.177935 2996 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ranger-kms
I20260620 23:00:59.178081 2996 mini_ranger_kms.cc:227] Using postgres at 127.2.237.20:48037
2026-06-20 23:00:59,599 [I] DB FLAVOR :POSTGRES
2026-06-20 23:00:59,600 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 23:00:59,600 [I] Checking connection
2026-06-20 23:00:59,600 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:48037/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-20 23:00:59,856 [I] connection success
2026-06-20 23:00:59,856 [I] --------- Verifying Ranger DB tables ---------
2026-06-20 23:00:59,856 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-20 23:00:59,857 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:48037/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-20 23:01:00,127 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-20 23:01:00,127 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 23:01:00,127 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-20 23:01:00,127 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:48037/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-20 23:01:00,439 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260620 23:01:00.938441 2996 mini_ranger_kms.cc:310] Created kms service
I20260620 23:01:01.143142 2996 mini_ranger_kms.cc:326] Created kudu user
I20260620 23:01:01.193480 2996 mini_ranger_kms.cc:343] Created rangerkms user
Jun 20 23:01:01 dist-test-slave-f63b krb5kdc[13832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996458, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.2.237.20@KRBTEST.COM
I20260620 23:01:01.623147 2996 mini_ranger_kms.cc:384] Added ranger policy
I20260620 23:01:01.623482 2996 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ranger-kms:/tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260620 23:01:01.623533 2996 mini_ranger_kms.cc:240] Using host: 127.2.237.20
I20260620 23:01:01.645216 2996 mini_ranger_kms.cc:276] Ranger KMS PID: 14226
I20260620 23:01:01.645428 2996 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.2.237.20:54793
Jun 20, 2026 11:01:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 11:01:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 11:01:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 20, 2026 11:01:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 20, 2026 11:01:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ranger-kms/ews/webapp].
Jun 20, 2026 11:01:03 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260620 23:01:03.941257 2996 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.681s user 0.085s sys 0.363s
I20260620 23:01:03.941532 2996 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260620 23:01:03.941761 2996 mini_ranger_kms.cc:401] 127.2.237.20:54793/kms/v1/keys
Jun 20 23:01:04 dist-test-slave-f63b krb5kdc[13832](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781996464, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.237.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 20 23:01:04 dist-test-slave-f63b krb5kdc[13832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996464, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.237.20@KRBTEST.COM for HTTP/127.2.237.20@KRBTEST.COM
Jun 20 23:01:04 dist-test-slave-f63b krb5kdc[13832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996464, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.237.20@KRBTEST.COM for HTTP/127.2.237.20@KRBTEST.COM
Jun 20 23:01:07 dist-test-slave-f63b krb5kdc[13832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
I20260620 23:01:07.290465 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:37797
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:37797
--ranger_config_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.237.20:54793/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:01:07.656442 14276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:01:07.656759 14276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:01:07.656898 14276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:01:07.665309 14276 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:01:07.665439 14276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:01:07.665515 14276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:01:07.665601 14276 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:01:07.665656 14276 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:01:07.676435 14276 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/master-0/wal
--ranger_kms_url=127.2.237.20:54793/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:37797
--ranger_config_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:37797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:01:07.677821 14276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:01:07.679294 14276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:01:07.693302 14281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:01:07.693411 14282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:01:07.694098 14284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:01:07.694641 14276 server_base.cc:1034] running on GCE node
I20260620 23:01:07.695657 14276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:01:07.697542 14276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:01:07.698788 14276 hybrid_clock.cc:648] HybridClock initialized: now 1781996467698724 us; error 59 us; skew 500 ppm
I20260620 23:01:07.699204 14276 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:01:07 dist-test-slave-f63b krb5kdc[13832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996467, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:01:07.706681 14276 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 23:01:07.708437 14276 webserver.cc:460] Webserver started at http://127.2.237.62:34919/ using document root <none> and password file <none>
I20260620 23:01:07.709188 14276 fs_manager.cc:359] Metadata directory not provided
I20260620 23:01:07.709290 14276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:01:07.709636 14276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 23:01:07 dist-test-slave-f63b krb5kdc[13832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996467, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for HTTP/127.2.237.20@KRBTEST.COM
I20260620 23:01:07.770064 14276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "4e4974e83b4143c29283c00beb50561f"
format_stamp: "Formatted at 2026-06-20 23:01:07 on dist-test-slave-f63b"
server_key: "d9b7946af7cd8228fb1d4b4b449a0589"
server_key_iv: "decfd064b824ef5c7ed04ceb043eeac2"
server_key_version: "kuduclusterkey@0"
I20260620 23:01:07.771318 14276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "4e4974e83b4143c29283c00beb50561f"
format_stamp: "Formatted at 2026-06-20 23:01:07 on dist-test-slave-f63b"
server_key: "d9b7946af7cd8228fb1d4b4b449a0589"
server_key_iv: "decfd064b824ef5c7ed04ceb043eeac2"
server_key_version: "kuduclusterkey@0"
I20260620 23:01:07.778987 14276 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260620 23:01:07.798043 14291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:01:07.799933 14276 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260620 23:01:07.800158 14276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "4e4974e83b4143c29283c00beb50561f"
format_stamp: "Formatted at 2026-06-20 23:01:07 on dist-test-slave-f63b"
server_key: "d9b7946af7cd8228fb1d4b4b449a0589"
server_key_iv: "decfd064b824ef5c7ed04ceb043eeac2"
server_key_version: "kuduclusterkey@0"
I20260620 23:01:07.800382 14276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:01:07.816394 14276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:01:07.820786 14276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:01:07.821116 14276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:01:07.842556 14276 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:37797
I20260620 23:01:07.842602 14342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:37797 every 8 connection(s)
I20260620 23:01:07.844038 14276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:01:07.848249 14343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:01:07.849727 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 14276
I20260620 23:01:07.849946 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:01:07.850735 2996 external_mini_cluster.cc:1428] Setting key f39dbe40dde7a802d13761616eb02fa3
I20260620 23:01:07.867309 14343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e4974e83b4143c29283c00beb50561f: Bootstrap starting.
I20260620 23:01:07.871891 14343 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e4974e83b4143c29283c00beb50561f: Neither blocks nor log segments found. Creating new log.
I20260620 23:01:07.873260 14343 log.cc:824] T 00000000000000000000000000000000 P 4e4974e83b4143c29283c00beb50561f: Log is configured to *not* fsync() on all Append() calls
I20260620 23:01:07.876407 14343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e4974e83b4143c29283c00beb50561f: No bootstrap required, opened a new log
Jun 20 23:01:07 dist-test-slave-f63b krb5kdc[13832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:01:07.887490 14343 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e4974e83b4143c29283c00beb50561f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e4974e83b4143c29283c00beb50561f" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37797 } }
I20260620 23:01:07.888046 14343 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e4974e83b4143c29283c00beb50561f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:01:07.888156 14343 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e4974e83b4143c29283c00beb50561f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e4974e83b4143c29283c00beb50561f, State: Initialized, Role: FOLLOWER
I20260620 23:01:07.888897 14343 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e4974e83b4143c29283c00beb50561f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e4974e83b4143c29283c00beb50561f" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37797 } }
I20260620 23:01:07.889189 14343 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e4974e83b4143c29283c00beb50561f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:01:07.889351 14343 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e4974e83b4143c29283c00beb50561f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:01:07.889590 14343 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e4974e83b4143c29283c00beb50561f [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:01:07.892983 14343 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e4974e83b4143c29283c00beb50561f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e4974e83b4143c29283c00beb50561f" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37797 } }
I20260620 23:01:07.893186 14346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:01:07.852955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33150 (local address 127.2.237.62:37797)
0620 23:01:07.867847 (+ 14892us) server_negotiation.cc:206] Beginning negotiation
0620 23:01:07.867865 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0620 23:01:07.867920 (+ 55us) server_negotiation.cc:407] Connection header received
0620 23:01:07.870595 (+ 2675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:01:07.870630 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:01:07.871182 (+ 552us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:01:07.871646 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:01:07.873064 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:01:07.874925 (+ 1861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:01:07.876649 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:01:07.877646 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:01:07.877857 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:01:07.883215 (+ 5358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:01:07.883227 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:01:07.883252 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:01:07.883324 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:01:07.887738 (+ 4414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:01:07.888717 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:01:07.888726 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:01:07.888736 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:01:07.888933 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:01:07.889418 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:01:07.889425 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:01:07.889427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:01:07.890290 (+ 863us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:01:07.890644 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0620 23:01:07.891169 (+ 525us) server_negotiation.cc:299] Negotiation successful
0620 23:01:07.891636 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14605,"thread_start_us":14366,"threads_started":1}
I20260620 23:01:07.893664 14343 leader_election.cc:302] T 00000000000000000000000000000000 P 4e4974e83b4143c29283c00beb50561f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e4974e83b4143c29283c00beb50561f; no voters:
I20260620 23:01:07.894411 14343 leader_election.cc:288] T 00000000000000000000000000000000 P 4e4974e83b4143c29283c00beb50561f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:01:07.894527 14348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e4974e83b4143c29283c00beb50561f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:01:07.897419 14343 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e4974e83b4143c29283c00beb50561f [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:01:07.902925 14348 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e4974e83b4143c29283c00beb50561f [term 1 LEADER]: Becoming Leader. State: Replica: 4e4974e83b4143c29283c00beb50561f, State: Running, Role: LEADER
I20260620 23:01:07.903790 14348 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e4974e83b4143c29283c00beb50561f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e4974e83b4143c29283c00beb50561f" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37797 } }
I20260620 23:01:07.907516 14350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e4974e83b4143c29283c00beb50561f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e4974e83b4143c29283c00beb50561f. Latest consensus state: current_term: 1 leader_uuid: "4e4974e83b4143c29283c00beb50561f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e4974e83b4143c29283c00beb50561f" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37797 } } }
I20260620 23:01:07.907150 14349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e4974e83b4143c29283c00beb50561f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e4974e83b4143c29283c00beb50561f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e4974e83b4143c29283c00beb50561f" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37797 } } }
I20260620 23:01:07.916532 14350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e4974e83b4143c29283c00beb50561f [sys.catalog]: This master's current role is: LEADER
I20260620 23:01:07.916654 14349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e4974e83b4143c29283c00beb50561f [sys.catalog]: This master's current role is: LEADER
I20260620 23:01:07.923576 14357 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:01:07.932952 14357 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:01:07.934084 14343 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260620 23:01:07.945588 14357 catalog_manager.cc:1347] Generated new cluster ID: ab3ce847a19940da86520b90eb961285
I20260620 23:01:07.945744 14357 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:01:08.008040 14357 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:01:08.011781 14357 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:01:08.020907 14357 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e4974e83b4143c29283c00beb50561f: Generated new TSK 0
I20260620 23:01:08.022328 14357 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.237.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
I20260620 23:01:08.952623 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.237.20:54793/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:0
--local_ip_for_outbound_sockets=127.2.237.1
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:37797
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 23:01:09 dist-test-slave-f63b krb5kdc[13832](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781996469, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260620 23:01:09.356456 14388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:01:09.356855 14388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:01:09.357016 14388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:01:09.365388 14388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:01:09.365557 14388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:01:09.365749 14388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 23:01:09.376714 14388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.2.237.20:54793/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:37797
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:01:09.378126 14388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:01:09.379527 14388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:01:09.398368 14398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:01:09.400691 14399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:01:09.409951 14388 server_base.cc:1034] running on GCE node
W20260620 23:01:09.410357 14401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:01:09.411167 14388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:01:09.412330 14388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:01:09.413614 14388 hybrid_clock.cc:648] HybridClock initialized: now 1781996469413497 us; error 105 us; skew 500 ppm
I20260620 23:01:09.414085 14388 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:01:09 dist-test-slave-f63b krb5kdc[13832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996469, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:01:09.418246 14388 init.cc:377] Logged in from keytab as kudu/127.2.237.1@KRBTEST.COM (short username kudu)
I20260620 23:01:09.423902 14388 webserver.cc:460] Webserver started at http://127.2.237.1:35011/ using document root <none> and password file <none>
I20260620 23:01:09.425060 14388 fs_manager.cc:359] Metadata directory not provided
I20260620 23:01:09.425253 14388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:01:09.425707 14388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 23:01:09 dist-test-slave-f63b krb5kdc[13832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996469, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for HTTP/127.2.237.20@KRBTEST.COM
I20260620 23:01:09.456614 14388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-0/data/instance:
uuid: "d48efd718bcf47f9943ed05c5acb9200"
format_stamp: "Formatted at 2026-06-20 23:01:09 on dist-test-slave-f63b"
server_key: "d4d7af050bfeababcabe5295c67e2750"
server_key_iv: "aa528a99cc55c43851420c8319d3325f"
server_key_version: "kuduclusterkey@0"
I20260620 23:01:09.457953 14388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance:
uuid: "d48efd718bcf47f9943ed05c5acb9200"
format_stamp: "Formatted at 2026-06-20 23:01:09 on dist-test-slave-f63b"
server_key: "d4d7af050bfeababcabe5295c67e2750"
server_key_iv: "aa528a99cc55c43851420c8319d3325f"
server_key_version: "kuduclusterkey@0"
I20260620 23:01:09.477345 14388 fs_manager.cc:691] Time spent creating directory manager: real 0.019s user 0.004s sys 0.007s
I20260620 23:01:09.504739 14408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:01:09.510208 14388 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.009s sys 0.000s
I20260620 23:01:09.510541 14388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "d48efd718bcf47f9943ed05c5acb9200"
format_stamp: "Formatted at 2026-06-20 23:01:09 on dist-test-slave-f63b"
server_key: "d4d7af050bfeababcabe5295c67e2750"
server_key_iv: "aa528a99cc55c43851420c8319d3325f"
server_key_version: "kuduclusterkey@0"
I20260620 23:01:09.510972 14388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:01:09.544341 14388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:01:09.549187 14388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:01:09.549510 14388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:01:09.550629 14388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:01:09.552248 14388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:01:09.552345 14388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:01:09.552527 14388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:01:09.552608 14388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 23:01:09 dist-test-slave-f63b krb5kdc[13832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996469, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for HTTP/127.2.237.20@KRBTEST.COM
I20260620 23:01:10.042315 14388 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:40477
I20260620 23:01:10.042380 14520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:40477 every 8 connection(s)
I20260620 23:01:10.044086 14388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 23:01:10.051678 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 14388
I20260620 23:01:10.051908 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance
I20260620 23:01:10.052578 2996 external_mini_cluster.cc:1428] Setting key fefd852f21d48181e09478bfec540d7a
Jun 20 23:01:10 dist-test-slave-f63b krb5kdc[13832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996469, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.237.2@KRBTEST.COM; defaulting to no policy
I20260620 23:01:10.087795 14524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:01:10.048323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:50481 (local address 127.2.237.62:37797)
0620 23:01:10.063323 (+ 15000us) server_negotiation.cc:206] Beginning negotiation
0620 23:01:10.063330 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:01:10.066681 (+ 3351us) server_negotiation.cc:407] Connection header received
0620 23:01:10.067058 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:01:10.067066 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:01:10.067171 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:01:10.067268 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:01:10.070733 (+ 3465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:01:10.071863 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:01:10.074268 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:01:10.074875 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:01:10.074974 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:01:10.079896 (+ 4922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:01:10.079903 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:01:10.079907 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:01:10.079949 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:01:10.083596 (+ 3647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:01:10.084704 (+ 1108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:01:10.084710 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:01:10.084714 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:01:10.084869 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:01:10.085444 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:01:10.085449 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:01:10.085452 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:01:10.085822 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:01:10.085956 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0620 23:01:10.087123 (+ 1167us) server_negotiation.cc:299] Negotiation successful
0620 23:01:10.087375 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14844,"thread_start_us":14692,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.2@KRBTEST.COM" created.
I20260620 23:01:10.091159 14523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:01:10.047665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37797 (local address 127.2.237.1:50481)
0620 23:01:10.064013 (+ 16348us) negotiation.cc:107] Waiting for socket to connect
0620 23:01:10.064095 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0620 23:01:10.066670 (+ 2575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:01:10.067902 (+ 1232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:01:10.067916 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:01:10.068590 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:01:10.069515 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:01:10.069544 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:01:10.072088 (+ 2544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:01:10.072093 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:01:10.074085 (+ 1992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:01:10.074102 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:01:10.075379 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:01:10.075385 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:01:10.075665 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:01:10.076496 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:01:10.076526 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:01:10.079611 (+ 3085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:01:10.083824 (+ 4213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:01:10.083834 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:01:10.083857 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:01:10.084550 (+ 693us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:01:10.085021 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:01:10.085025 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:01:10.085028 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:01:10.085311 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:01:10.086062 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:01:10.086071 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:01:10.086551 (+ 480us) client_negotiation.cc:769] Sending connection context
0620 23:01:10.086953 (+ 402us) client_negotiation.cc:240] Negotiation successful
0620 23:01:10.087548 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15926,"thread_start_us":15693,"threads_started":1}
I20260620 23:01:10.093827 14521 heartbeater.cc:347] Connected to a master server at 127.2.237.62:37797
I20260620 23:01:10.094173 14521 heartbeater.cc:464] Registering TS with master...
I20260620 23:01:10.095177 14521 heartbeater.cc:511] Master 127.2.237.62:37797 requested a full tablet report, sending...
I20260620 23:01:10.097782 14308 ts_manager.cc:194] Registered new tserver with Master: d48efd718bcf47f9943ed05c5acb9200 (127.2.237.1:40477)
I20260620 23:01:10.099752 14308 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.1@KRBTEST.COM'} at 127.2.237.1:50481
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
I20260620 23:01:10.125213 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.237.20:54793/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:0
--local_ip_for_outbound_sockets=127.2.237.2
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:37797
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 23:01:10 dist-test-slave-f63b krb5kdc[13832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996469, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for HTTP/127.2.237.20@KRBTEST.COM
W20260620 23:01:10.622166 14528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:01:10.622614 14528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:01:10.622777 14528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:01:10.633802 14528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:01:10.633973 14528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:01:10.634166 14528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 23:01:10.644579 14528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.2.237.20:54793/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:37797
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:01:10.645991 14528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:01:10.647455 14528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:01:10.661017 14542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:01:10.661172 14541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:01:10.661824 14528 server_base.cc:1034] running on GCE node
W20260620 23:01:10.662326 14544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:01:10.663048 14528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:01:10.664177 14528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:01:10.665426 14528 hybrid_clock.cc:648] HybridClock initialized: now 1781996470665372 us; error 50 us; skew 500 ppm
I20260620 23:01:10.666154 14528 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:01:10 dist-test-slave-f63b krb5kdc[13832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996470, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:01:10.669437 14528 init.cc:377] Logged in from keytab as kudu/127.2.237.2@KRBTEST.COM (short username kudu)
I20260620 23:01:10.671200 14528 webserver.cc:460] Webserver started at http://127.2.237.2:45539/ using document root <none> and password file <none>
I20260620 23:01:10.671933 14528 fs_manager.cc:359] Metadata directory not provided
I20260620 23:01:10.672062 14528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:01:10.672449 14528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 23:01:10 dist-test-slave-f63b krb5kdc[13832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996470, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for HTTP/127.2.237.20@KRBTEST.COM
I20260620 23:01:10.690872 14528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-1/data/instance:
uuid: "f1a60b1cc5154450aba87ca0f26ccab0"
format_stamp: "Formatted at 2026-06-20 23:01:10 on dist-test-slave-f63b"
server_key: "25e355cfb95a63737253cfd35ae85922"
server_key_iv: "dcf12c21701f4d93a0813cd28abccea4"
server_key_version: "kuduclusterkey@0"
I20260620 23:01:10.692062 14528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance:
uuid: "f1a60b1cc5154450aba87ca0f26ccab0"
format_stamp: "Formatted at 2026-06-20 23:01:10 on dist-test-slave-f63b"
server_key: "25e355cfb95a63737253cfd35ae85922"
server_key_iv: "dcf12c21701f4d93a0813cd28abccea4"
server_key_version: "kuduclusterkey@0"
I20260620 23:01:10.699908 14528 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260620 23:01:10.724953 14551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:01:10.727144 14528 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260620 23:01:10.727355 14528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "f1a60b1cc5154450aba87ca0f26ccab0"
format_stamp: "Formatted at 2026-06-20 23:01:10 on dist-test-slave-f63b"
server_key: "25e355cfb95a63737253cfd35ae85922"
server_key_iv: "dcf12c21701f4d93a0813cd28abccea4"
server_key_version: "kuduclusterkey@0"
I20260620 23:01:10.727653 14528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:01:10.758402 14528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:01:10.764460 14528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:01:10.764813 14528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:01:10.766026 14528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:01:10.767861 14528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:01:10.767954 14528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:01:10.768121 14528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:01:10.768179 14528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:01:10.814716 14528 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:45193
I20260620 23:01:10.814755 14663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:45193 every 8 connection(s)
I20260620 23:01:10.816645 14528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 23:01:10.821400 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 14528
I20260620 23:01:10.821574 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance
I20260620 23:01:10.822070 2996 external_mini_cluster.cc:1428] Setting key 0fc97fe5937049595879e5f970c27308
WARNING: no policy specified for kudu/127.2.237.3@KRBTEST.COM; defaulting to no policy
Jun 20 23:01:10 dist-test-slave-f63b krb5kdc[13832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996470, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.3@KRBTEST.COM" created.
I20260620 23:01:10.868232 14666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:01:10.820063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:48575 (local address 127.2.237.62:37797)
0620 23:01:10.842666 (+ 22603us) server_negotiation.cc:206] Beginning negotiation
0620 23:01:10.842686 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0620 23:01:10.842709 (+ 23us) server_negotiation.cc:407] Connection header received
0620 23:01:10.842776 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:01:10.842782 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:01:10.842871 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:01:10.842983 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:01:10.845322 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:01:10.846505 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:01:10.849199 (+ 2694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:01:10.849874 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:01:10.849976 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:01:10.860428 (+ 10452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:01:10.860438 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:01:10.860443 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:01:10.860494 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:01:10.863996 (+ 3502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:01:10.865099 (+ 1103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:01:10.865105 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:01:10.865109 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:01:10.865290 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:01:10.865918 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:01:10.865923 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:01:10.865926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:01:10.866312 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:01:10.866434 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0620 23:01:10.867577 (+ 1143us) server_negotiation.cc:299] Negotiation successful
0620 23:01:10.867837 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22401,"thread_start_us":23037,"threads_started":1}
I20260620 23:01:10.870609 14667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:01:10.820716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37797 (local address 127.2.237.2:48575)
0620 23:01:10.835871 (+ 15155us) negotiation.cc:107] Waiting for socket to connect
0620 23:01:10.835955 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0620 23:01:10.841243 (+ 5288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:01:10.843571 (+ 2328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:01:10.843585 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:01:10.844252 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:01:10.845094 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:01:10.845120 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:01:10.846801 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:01:10.846807 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:01:10.849012 (+ 2205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:01:10.849029 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:01:10.850358 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:01:10.850365 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:01:10.850737 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:01:10.855963 (+ 5226us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:01:10.856004 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:01:10.860066 (+ 4062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:01:10.864234 (+ 4168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:01:10.864246 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:01:10.864269 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:01:10.864931 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:01:10.865452 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:01:10.865457 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:01:10.865461 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:01:10.865744 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:01:10.866542 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:01:10.866563 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:01:10.867083 (+ 520us) client_negotiation.cc:769] Sending connection context
0620 23:01:10.868568 (+ 1485us) client_negotiation.cc:240] Negotiation successful
0620 23:01:10.869153 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14691,"thread_start_us":14456,"threads_started":1}
I20260620 23:01:10.872344 14664 heartbeater.cc:347] Connected to a master server at 127.2.237.62:37797
I20260620 23:01:10.872692 14664 heartbeater.cc:464] Registering TS with master...
I20260620 23:01:10.873575 14664 heartbeater.cc:511] Master 127.2.237.62:37797 requested a full tablet report, sending...
I20260620 23:01:10.875303 14308 ts_manager.cc:194] Registered new tserver with Master: f1a60b1cc5154450aba87ca0f26ccab0 (127.2.237.2:45193)
I20260620 23:01:10.876420 14308 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.2@KRBTEST.COM'} at 127.2.237.2:48575
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
I20260620 23:01:10.899575 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.237.20:54793/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:0
--local_ip_for_outbound_sockets=127.2.237.3
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:37797
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:38197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:01:11.104970 14521 heartbeater.cc:503] Master 127.2.237.62:37797 was elected leader, sending a full tablet report...
W20260620 23:01:11.268570 14670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:01:11.268949 14670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:01:11.269119 14670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:01:11.277251 14670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:01:11.277438 14670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:01:11.277627 14670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 23:01:11.288342 14670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:38197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.2.237.20:54793/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:37797
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:01:11.289803 14670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:01:11.291765 14670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:01:11.307189 14676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:01:11.307765 14675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:01:11.307806 14678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:01:11.308322 14670 server_base.cc:1034] running on GCE node
I20260620 23:01:11.309032 14670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:01:11.310316 14670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:01:11.311595 14670 hybrid_clock.cc:648] HybridClock initialized: now 1781996471311519 us; error 64 us; skew 500 ppm
I20260620 23:01:11.311987 14670 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:01:11 dist-test-slave-f63b krb5kdc[13832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996471, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:01:11.315661 14670 init.cc:377] Logged in from keytab as kudu/127.2.237.3@KRBTEST.COM (short username kudu)
I20260620 23:01:11.318004 14670 webserver.cc:460] Webserver started at http://127.2.237.3:40475/ using document root <none> and password file <none>
I20260620 23:01:11.318918 14670 fs_manager.cc:359] Metadata directory not provided
I20260620 23:01:11.319067 14670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:01:11.319399 14670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 23:01:11 dist-test-slave-f63b krb5kdc[13832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996471, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for HTTP/127.2.237.20@KRBTEST.COM
I20260620 23:01:11.336251 14670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-2/data/instance:
uuid: "4ead0729b1774788972222d3363ce7d8"
format_stamp: "Formatted at 2026-06-20 23:01:11 on dist-test-slave-f63b"
server_key: "0bd8fdf3531bb164dd7f25df7133814a"
server_key_iv: "9b1688591dfb6a437de2cb8d1a7386a2"
server_key_version: "kuduclusterkey@0"
I20260620 23:01:11.337162 14670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance:
uuid: "4ead0729b1774788972222d3363ce7d8"
format_stamp: "Formatted at 2026-06-20 23:01:11 on dist-test-slave-f63b"
server_key: "0bd8fdf3531bb164dd7f25df7133814a"
server_key_iv: "9b1688591dfb6a437de2cb8d1a7386a2"
server_key_version: "kuduclusterkey@0"
I20260620 23:01:11.343101 14670 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260620 23:01:11.360469 14685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:01:11.362394 14670 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260620 23:01:11.362609 14670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "4ead0729b1774788972222d3363ce7d8"
format_stamp: "Formatted at 2026-06-20 23:01:11 on dist-test-slave-f63b"
server_key: "0bd8fdf3531bb164dd7f25df7133814a"
server_key_iv: "9b1688591dfb6a437de2cb8d1a7386a2"
server_key_version: "kuduclusterkey@0"
I20260620 23:01:11.362865 14670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:01:11.421794 14670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:01:11.426553 14670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:01:11.426946 14670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:01:11.428071 14670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:01:11.429592 14670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:01:11.429703 14670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:01:11.429850 14670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:01:11.429919 14670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:01:11.469210 14670 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:33089
I20260620 23:01:11.469230 14797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:33089 every 8 connection(s)
I20260620 23:01:11.470716 14670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 23:01:11.471602 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 14670
I20260620 23:01:11.471777 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance
I20260620 23:01:11.472313 2996 external_mini_cluster.cc:1428] Setting key 21f2d7d979319b4ef7550ff55b19ab60
Jun 20 23:01:11 dist-test-slave-f63b krb5kdc[13832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996471, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:01:11.497643 14800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:01:11.474447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:47277 (local address 127.2.237.62:37797)
0620 23:01:11.476745 (+ 2298us) server_negotiation.cc:206] Beginning negotiation
0620 23:01:11.476751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:01:11.477294 (+ 543us) server_negotiation.cc:407] Connection header received
0620 23:01:11.479833 (+ 2539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:01:11.479838 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:01:11.479914 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:01:11.480005 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:01:11.482076 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:01:11.482957 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:01:11.485500 (+ 2543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:01:11.485952 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:01:11.486105 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:01:11.490105 (+ 4000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:01:11.490112 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:01:11.490118 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:01:11.490160 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:01:11.494046 (+ 3886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:01:11.495072 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:01:11.495083 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:01:11.495087 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:01:11.495245 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:01:11.495681 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:01:11.495686 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:01:11.495690 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:01:11.496064 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:01:11.496220 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0620 23:01:11.497012 (+ 792us) server_negotiation.cc:299] Negotiation successful
0620 23:01:11.497253 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2143,"thread_start_us":1955,"threads_started":1}
I20260620 23:01:11.498474 14801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:01:11.475222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37797 (local address 127.2.237.3:47277)
0620 23:01:11.477114 (+ 1892us) negotiation.cc:107] Waiting for socket to connect
0620 23:01:11.477165 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0620 23:01:11.479508 (+ 2343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:01:11.480327 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:01:11.480341 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:01:11.480981 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:01:11.481856 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:01:11.481883 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:01:11.483163 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:01:11.483169 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:01:11.485321 (+ 2152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:01:11.485337 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:01:11.486238 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:01:11.486246 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:01:11.486555 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:01:11.487388 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:01:11.487420 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:01:11.489866 (+ 2446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:01:11.494249 (+ 4383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:01:11.494262 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:01:11.494286 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:01:11.494937 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:01:11.495357 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:01:11.495361 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:01:11.495363 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:01:11.495567 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:01:11.496221 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:01:11.496228 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:01:11.496607 (+ 379us) client_negotiation.cc:769] Sending connection context
0620 23:01:11.496864 (+ 257us) client_negotiation.cc:240] Negotiation successful
0620 23:01:11.497286 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1575,"thread_start_us":1366,"threads_started":1}
I20260620 23:01:11.500852 14798 heartbeater.cc:347] Connected to a master server at 127.2.237.62:37797
I20260620 23:01:11.501173 14798 heartbeater.cc:464] Registering TS with master...
I20260620 23:01:11.501873 14798 heartbeater.cc:511] Master 127.2.237.62:37797 requested a full tablet report, sending...
I20260620 23:01:11.503475 14307 ts_manager.cc:194] Registered new tserver with Master: 4ead0729b1774788972222d3363ce7d8 (127.2.237.3:33089)
I20260620 23:01:11.504767 14307 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.3@KRBTEST.COM'} at 127.2.237.3:47277
I20260620 23:01:11.514035 2996 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:01:11.533953 14800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:01:11.519964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33162 (local address 127.2.237.62:37797)
0620 23:01:11.520175 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0620 23:01:11.520182 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:01:11.520726 (+ 544us) server_negotiation.cc:407] Connection header received
0620 23:01:11.520882 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:01:11.520888 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:01:11.520986 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:01:11.521114 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:01:11.522607 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:01:11.523657 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:01:11.525039 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:01:11.525690 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:01:11.525824 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:01:11.528838 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:01:11.528843 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:01:11.528847 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:01:11.528881 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:01:11.531546 (+ 2665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:01:11.532217 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:01:11.532221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:01:11.532223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:01:11.532350 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:01:11.532682 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:01:11.532685 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:01:11.532687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:01:11.532963 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:01:11.533147 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0620 23:01:11.533500 (+ 353us) server_negotiation.cc:299] Negotiation successful
0620 23:01:11.533670 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260620 23:01:11.538230 14307 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33162:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:01:11.545295 14307 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:01:11.569919 14812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:01:11.559141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47872 (local address 127.2.237.3:33089)
0620 23:01:11.560982 (+ 1841us) server_negotiation.cc:206] Beginning negotiation
0620 23:01:11.560992 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:01:11.561022 (+ 30us) server_negotiation.cc:407] Connection header received
0620 23:01:11.561123 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:01:11.561132 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:01:11.561428 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:01:11.562650 (+ 1222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:01:11.562990 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:01:11.564898 (+ 1908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:01:11.567389 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:01:11.568805 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:01:11.568934 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:01:11.569227 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0620 23:01:11.569404 (+ 177us) server_negotiation.cc:299] Negotiation successful
0620 23:01:11.569580 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1628,"thread_start_us":1482,"threads_started":1}
I20260620 23:01:11.570003 14813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:01:11.558998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:33089 (local address 127.0.0.1:47872)
0620 23:01:11.559998 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0620 23:01:11.560049 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0620 23:01:11.560284 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:01:11.561698 (+ 1414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:01:11.561708 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:01:11.561753 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:01:11.562345 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:01:11.562364 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:01:11.565022 (+ 2658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:01:11.565027 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:01:11.567186 (+ 2159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:01:11.567225 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:01:11.568972 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:01:11.568978 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:01:11.569163 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:01:11.569215 (+ 52us) client_negotiation.cc:769] Sending connection context
0620 23:01:11.569350 (+ 135us) client_negotiation.cc:240] Negotiation successful
0620 23:01:11.569535 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"mutex_wait_us":1,"thread_start_us":1967,"threads_started":1}
I20260620 23:01:11.570988 14811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:01:11.559210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:40477 (local address 127.0.0.1:46826)
0620 23:01:11.561273 (+ 2063us) negotiation.cc:107] Waiting for socket to connect
0620 23:01:11.561296 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0620 23:01:11.561404 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:01:11.563600 (+ 2196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:01:11.563606 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:01:11.563630 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:01:11.563970 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:01:11.563981 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:01:11.565937 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:01:11.565942 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:01:11.567978 (+ 2036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:01:11.567992 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:01:11.569824 (+ 1832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:01:11.569830 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:01:11.570418 (+ 588us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:01:11.570458 (+ 40us) client_negotiation.cc:769] Sending connection context
0620 23:01:11.570534 (+ 76us) client_negotiation.cc:240] Negotiation successful
0620 23:01:11.570678 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1932,"thread_start_us":391,"threads_started":1}
I20260620 23:01:11.571339 14814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:01:11.559866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46826 (local address 127.2.237.1:40477)
0620 23:01:11.562995 (+ 3129us) server_negotiation.cc:206] Beginning negotiation
0620 23:01:11.563007 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 23:01:11.563052 (+ 45us) server_negotiation.cc:407] Connection header received
0620 23:01:11.563159 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:01:11.563167 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:01:11.563399 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:01:11.563598 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:01:11.564163 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:01:11.565790 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:01:11.568158 (+ 2368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:01:11.569608 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:01:11.570485 (+ 877us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:01:11.570753 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0620 23:01:11.570829 (+ 76us) server_negotiation.cc:299] Negotiation successful
0620 23:01:11.570996 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2965,"thread_start_us":2829,"threads_started":1}
I20260620 23:01:11.572114 14815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:01:11.561362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:45193 (local address 127.0.0.1:48694)
0620 23:01:11.562266 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0620 23:01:11.562289 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0620 23:01:11.562426 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:01:11.564407 (+ 1981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:01:11.564414 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:01:11.564439 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:01:11.564792 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:01:11.564802 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:01:11.566693 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:01:11.566699 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:01:11.568741 (+ 2042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:01:11.568754 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:01:11.570958 (+ 2204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:01:11.570963 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:01:11.571140 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:01:11.571173 (+ 33us) client_negotiation.cc:769] Sending connection context
0620 23:01:11.571259 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 23:01:11.571388 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"thread_start_us":657,"threads_started":1}
I20260620 23:01:11.572950 14816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:01:11.561854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48694 (local address 127.2.237.2:45193)
0620 23:01:11.563261 (+ 1407us) server_negotiation.cc:206] Beginning negotiation
0620 23:01:11.563268 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:01:11.563291 (+ 23us) server_negotiation.cc:407] Connection header received
0620 23:01:11.563372 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:01:11.563379 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:01:11.563984 (+ 605us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:01:11.564173 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:01:11.564953 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:01:11.566507 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:01:11.568883 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:01:11.570506 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:01:11.570650 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:01:11.570814 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0620 23:01:11.571659 (+ 845us) server_negotiation.cc:299] Negotiation successful
0620 23:01:11.572478 (+ 819us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1277,"thread_start_us":1166,"threads_started":1}
I20260620 23:01:11.574813 14733 tablet_service.cc:1467] Processing CreateTablet for tablet 330c67dc8e4c4b31ae1ae08b4bcac293 (DEFAULT_TABLE table=test-table [id=7db6202308f64da6a2b36ed049f8bfac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:01:11.575366 14456 tablet_service.cc:1467] Processing CreateTablet for tablet 330c67dc8e4c4b31ae1ae08b4bcac293 (DEFAULT_TABLE table=test-table [id=7db6202308f64da6a2b36ed049f8bfac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:01:11.575654 14599 tablet_service.cc:1467] Processing CreateTablet for tablet 330c67dc8e4c4b31ae1ae08b4bcac293 (DEFAULT_TABLE table=test-table [id=7db6202308f64da6a2b36ed049f8bfac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:01:11.576391 14733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 330c67dc8e4c4b31ae1ae08b4bcac293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:01:11.576761 14456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 330c67dc8e4c4b31ae1ae08b4bcac293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:01:11.576900 14599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 330c67dc8e4c4b31ae1ae08b4bcac293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:01:11.588375 14817 tablet_bootstrap.cc:492] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0: Bootstrap starting.
I20260620 23:01:11.592468 14818 tablet_bootstrap.cc:492] T 330c67dc8e4c4b31ae1ae08b4bcac293 P d48efd718bcf47f9943ed05c5acb9200: Bootstrap starting.
I20260620 23:01:11.592919 14817 tablet_bootstrap.cc:654] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0: Neither blocks nor log segments found. Creating new log.
I20260620 23:01:11.594340 14819 tablet_bootstrap.cc:492] T 330c67dc8e4c4b31ae1ae08b4bcac293 P 4ead0729b1774788972222d3363ce7d8: Bootstrap starting.
I20260620 23:01:11.594475 14817 log.cc:824] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0: Log is configured to *not* fsync() on all Append() calls
I20260620 23:01:11.597551 14817 tablet_bootstrap.cc:492] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0: No bootstrap required, opened a new log
I20260620 23:01:11.597824 14817 ts_tablet_manager.cc:1397] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260620 23:01:11.598143 14818 tablet_bootstrap.cc:654] T 330c67dc8e4c4b31ae1ae08b4bcac293 P d48efd718bcf47f9943ed05c5acb9200: Neither blocks nor log segments found. Creating new log.
I20260620 23:01:11.598181 14819 tablet_bootstrap.cc:654] T 330c67dc8e4c4b31ae1ae08b4bcac293 P 4ead0729b1774788972222d3363ce7d8: Neither blocks nor log segments found. Creating new log.
I20260620 23:01:11.599928 14818 log.cc:824] T 330c67dc8e4c4b31ae1ae08b4bcac293 P d48efd718bcf47f9943ed05c5acb9200: Log is configured to *not* fsync() on all Append() calls
I20260620 23:01:11.599928 14819 log.cc:824] T 330c67dc8e4c4b31ae1ae08b4bcac293 P 4ead0729b1774788972222d3363ce7d8: Log is configured to *not* fsync() on all Append() calls
I20260620 23:01:11.605334 14818 tablet_bootstrap.cc:492] T 330c67dc8e4c4b31ae1ae08b4bcac293 P d48efd718bcf47f9943ed05c5acb9200: No bootstrap required, opened a new log
I20260620 23:01:11.605409 14819 tablet_bootstrap.cc:492] T 330c67dc8e4c4b31ae1ae08b4bcac293 P 4ead0729b1774788972222d3363ce7d8: No bootstrap required, opened a new log
I20260620 23:01:11.605654 14818 ts_tablet_manager.cc:1397] T 330c67dc8e4c4b31ae1ae08b4bcac293 P d48efd718bcf47f9943ed05c5acb9200: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.005s
I20260620 23:01:11.605690 14819 ts_tablet_manager.cc:1397] T 330c67dc8e4c4b31ae1ae08b4bcac293 P 4ead0729b1774788972222d3363ce7d8: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.005s
I20260620 23:01:11.607317 14817 raft_consensus.cc:348] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ead0729b1774788972222d3363ce7d8" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 33089 } } peers { permanent_uuid: "d48efd718bcf47f9943ed05c5acb9200" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 40477 } } peers { permanent_uuid: "f1a60b1cc5154450aba87ca0f26ccab0" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 45193 } }
I20260620 23:01:11.608250 14817 raft_consensus.cc:374] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:01:11.608347 14817 raft_consensus.cc:729] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1a60b1cc5154450aba87ca0f26ccab0, State: Initialized, Role: FOLLOWER
I20260620 23:01:11.609158 14817 consensus_queue.cc:260] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ead0729b1774788972222d3363ce7d8" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 33089 } } peers { permanent_uuid: "d48efd718bcf47f9943ed05c5acb9200" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 40477 } } peers { permanent_uuid: "f1a60b1cc5154450aba87ca0f26ccab0" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 45193 } }
I20260620 23:01:11.610374 14664 heartbeater.cc:503] Master 127.2.237.62:37797 was elected leader, sending a full tablet report...
I20260620 23:01:11.611595 14817 ts_tablet_manager.cc:1428] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
I20260620 23:01:11.615876 14818 raft_consensus.cc:348] T 330c67dc8e4c4b31ae1ae08b4bcac293 P d48efd718bcf47f9943ed05c5acb9200 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ead0729b1774788972222d3363ce7d8" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 33089 } } peers { permanent_uuid: "d48efd718bcf47f9943ed05c5acb9200" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 40477 } } peers { permanent_uuid: "f1a60b1cc5154450aba87ca0f26ccab0" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 45193 } }
I20260620 23:01:11.616420 14818 raft_consensus.cc:374] T 330c67dc8e4c4b31ae1ae08b4bcac293 P d48efd718bcf47f9943ed05c5acb9200 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:01:11.616555 14818 raft_consensus.cc:729] T 330c67dc8e4c4b31ae1ae08b4bcac293 P d48efd718bcf47f9943ed05c5acb9200 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d48efd718bcf47f9943ed05c5acb9200, State: Initialized, Role: FOLLOWER
I20260620 23:01:11.617008 14819 raft_consensus.cc:348] T 330c67dc8e4c4b31ae1ae08b4bcac293 P 4ead0729b1774788972222d3363ce7d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ead0729b1774788972222d3363ce7d8" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 33089 } } peers { permanent_uuid: "d48efd718bcf47f9943ed05c5acb9200" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 40477 } } peers { permanent_uuid: "f1a60b1cc5154450aba87ca0f26ccab0" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 45193 } }
I20260620 23:01:11.617486 14819 raft_consensus.cc:374] T 330c67dc8e4c4b31ae1ae08b4bcac293 P 4ead0729b1774788972222d3363ce7d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:01:11.617590 14819 raft_consensus.cc:729] T 330c67dc8e4c4b31ae1ae08b4bcac293 P 4ead0729b1774788972222d3363ce7d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ead0729b1774788972222d3363ce7d8, State: Initialized, Role: FOLLOWER
I20260620 23:01:11.617355 14818 consensus_queue.cc:260] T 330c67dc8e4c4b31ae1ae08b4bcac293 P d48efd718bcf47f9943ed05c5acb9200 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ead0729b1774788972222d3363ce7d8" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 33089 } } peers { permanent_uuid: "d48efd718bcf47f9943ed05c5acb9200" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 40477 } } peers { permanent_uuid: "f1a60b1cc5154450aba87ca0f26ccab0" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 45193 } }
I20260620 23:01:11.618227 14819 consensus_queue.cc:260] T 330c67dc8e4c4b31ae1ae08b4bcac293 P 4ead0729b1774788972222d3363ce7d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ead0729b1774788972222d3363ce7d8" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 33089 } } peers { permanent_uuid: "d48efd718bcf47f9943ed05c5acb9200" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 40477 } } peers { permanent_uuid: "f1a60b1cc5154450aba87ca0f26ccab0" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 45193 } }
I20260620 23:01:11.619381 14798 heartbeater.cc:503] Master 127.2.237.62:37797 was elected leader, sending a full tablet report...
I20260620 23:01:11.619874 14819 ts_tablet_manager.cc:1428] T 330c67dc8e4c4b31ae1ae08b4bcac293 P 4ead0729b1774788972222d3363ce7d8: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260620 23:01:11.620690 14818 ts_tablet_manager.cc:1428] T 330c67dc8e4c4b31ae1ae08b4bcac293 P d48efd718bcf47f9943ed05c5acb9200: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
W20260620 23:01:11.724174 14799 tablet.cc:2369] T 330c67dc8e4c4b31ae1ae08b4bcac293 P 4ead0729b1774788972222d3363ce7d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:01:11.802907 14522 tablet.cc:2369] T 330c67dc8e4c4b31ae1ae08b4bcac293 P d48efd718bcf47f9943ed05c5acb9200: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:01:11.821718 14823 raft_consensus.cc:482] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:01:11.822053 14823 raft_consensus.cc:504] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ead0729b1774788972222d3363ce7d8" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 33089 } } peers { permanent_uuid: "d48efd718bcf47f9943ed05c5acb9200" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 40477 } } peers { permanent_uuid: "f1a60b1cc5154450aba87ca0f26ccab0" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 45193 } }
I20260620 23:01:11.823961 14823 leader_election.cc:288] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ead0729b1774788972222d3363ce7d8 (127.2.237.3:33089), d48efd718bcf47f9943ed05c5acb9200 (127.2.237.1:40477)
I20260620 23:01:11.833058 14812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:01:11.824564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:50985 (local address 127.2.237.3:33089)
0620 23:01:11.825411 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0620 23:01:11.825418 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:01:11.825529 (+ 111us) server_negotiation.cc:407] Connection header received
0620 23:01:11.825861 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:01:11.825869 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:01:11.825959 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:01:11.826040 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:01:11.826883 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:01:11.827848 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:01:11.830410 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:01:11.831800 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:01:11.831961 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:01:11.832077 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0620 23:01:11.832661 (+ 584us) server_negotiation.cc:299] Negotiation successful
0620 23:01:11.832797 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":699}
I20260620 23:01:11.833117 14814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:01:11.825495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:54369 (local address 127.2.237.1:40477)
0620 23:01:11.825696 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0620 23:01:11.825704 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:01:11.826525 (+ 821us) server_negotiation.cc:407] Connection header received
0620 23:01:11.826712 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:01:11.826717 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:01:11.826792 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:01:11.826902 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:01:11.827484 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:01:11.828217 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:01:11.830593 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:01:11.831864 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:01:11.831989 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:01:11.832150 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0620 23:01:11.832661 (+ 511us) server_negotiation.cc:299] Negotiation successful
0620 23:01:11.832797 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260620 23:01:11.833371 14826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:01:11.824223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:33089 (local address 127.2.237.2:50985)
0620 23:01:11.825411 (+ 1188us) negotiation.cc:107] Waiting for socket to connect
0620 23:01:11.825431 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 23:01:11.825654 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:01:11.826280 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:01:11.826285 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:01:11.826313 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:01:11.826746 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:01:11.826756 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:01:11.827991 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:01:11.827997 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:01:11.830232 (+ 2235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:01:11.830248 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:01:11.832062 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:01:11.832068 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:01:11.832382 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:01:11.832521 (+ 139us) client_negotiation.cc:769] Sending connection context
0620 23:01:11.832621 (+ 100us) client_negotiation.cc:240] Negotiation successful
0620 23:01:11.832932 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1047,"mutex_wait_us":185,"thread_start_us":857,"threads_started":1}
I20260620 23:01:11.833412 14827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:01:11.824415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:40477 (local address 127.2.237.2:54369)
0620 23:01:11.826398 (+ 1983us) negotiation.cc:107] Waiting for socket to connect
0620 23:01:11.826423 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0620 23:01:11.826544 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:01:11.826922 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:01:11.826926 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:01:11.826956 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:01:11.827320 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:01:11.827330 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:01:11.828342 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:01:11.828347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:01:11.830413 (+ 2066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:01:11.830427 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:01:11.832073 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:01:11.832078 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:01:11.832384 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:01:11.832521 (+ 137us) client_negotiation.cc:769] Sending connection context
0620 23:01:11.832622 (+ 101us) client_negotiation.cc:240] Negotiation successful
0620 23:01:11.832887 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1852,"mutex_wait_us":175,"thread_start_us":1533,"threads_started":1}
I20260620 23:01:11.834805 14476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "330c67dc8e4c4b31ae1ae08b4bcac293" candidate_uuid: "f1a60b1cc5154450aba87ca0f26ccab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d48efd718bcf47f9943ed05c5acb9200" is_pre_election: true
I20260620 23:01:11.834805 14753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "330c67dc8e4c4b31ae1ae08b4bcac293" candidate_uuid: "f1a60b1cc5154450aba87ca0f26ccab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ead0729b1774788972222d3363ce7d8" is_pre_election: true
I20260620 23:01:11.835433 14753 raft_consensus.cc:2454] T 330c67dc8e4c4b31ae1ae08b4bcac293 P 4ead0729b1774788972222d3363ce7d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1a60b1cc5154450aba87ca0f26ccab0 in term 0.
I20260620 23:01:11.835433 14476 raft_consensus.cc:2454] T 330c67dc8e4c4b31ae1ae08b4bcac293 P d48efd718bcf47f9943ed05c5acb9200 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1a60b1cc5154450aba87ca0f26ccab0 in term 0.
I20260620 23:01:11.836212 14553 leader_election.cc:302] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ead0729b1774788972222d3363ce7d8, f1a60b1cc5154450aba87ca0f26ccab0; no voters:
I20260620 23:01:11.837057 14823 raft_consensus.cc:2781] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:01:11.837256 14823 raft_consensus.cc:482] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:01:11.837383 14823 raft_consensus.cc:3037] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0 [term 0 FOLLOWER]: Advancing to term 1
W20260620 23:01:11.837479 14665 tablet.cc:2369] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:01:11.839818 14823 raft_consensus.cc:504] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ead0729b1774788972222d3363ce7d8" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 33089 } } peers { permanent_uuid: "d48efd718bcf47f9943ed05c5acb9200" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 40477 } } peers { permanent_uuid: "f1a60b1cc5154450aba87ca0f26ccab0" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 45193 } }
I20260620 23:01:11.840557 14823 leader_election.cc:288] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0 [CANDIDATE]: Term 1 election: Requested vote from peers 4ead0729b1774788972222d3363ce7d8 (127.2.237.3:33089), d48efd718bcf47f9943ed05c5acb9200 (127.2.237.1:40477)
I20260620 23:01:11.841192 14476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "330c67dc8e4c4b31ae1ae08b4bcac293" candidate_uuid: "f1a60b1cc5154450aba87ca0f26ccab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d48efd718bcf47f9943ed05c5acb9200"
I20260620 23:01:11.841192 14753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "330c67dc8e4c4b31ae1ae08b4bcac293" candidate_uuid: "f1a60b1cc5154450aba87ca0f26ccab0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ead0729b1774788972222d3363ce7d8"
I20260620 23:01:11.841423 14753 raft_consensus.cc:3037] T 330c67dc8e4c4b31ae1ae08b4bcac293 P 4ead0729b1774788972222d3363ce7d8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:01:11.841423 14476 raft_consensus.cc:3037] T 330c67dc8e4c4b31ae1ae08b4bcac293 P d48efd718bcf47f9943ed05c5acb9200 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:01:11.844748 14476 raft_consensus.cc:2454] T 330c67dc8e4c4b31ae1ae08b4bcac293 P d48efd718bcf47f9943ed05c5acb9200 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1a60b1cc5154450aba87ca0f26ccab0 in term 1.
I20260620 23:01:11.844746 14753 raft_consensus.cc:2454] T 330c67dc8e4c4b31ae1ae08b4bcac293 P 4ead0729b1774788972222d3363ce7d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1a60b1cc5154450aba87ca0f26ccab0 in term 1.
I20260620 23:01:11.845283 14554 leader_election.cc:302] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d48efd718bcf47f9943ed05c5acb9200, f1a60b1cc5154450aba87ca0f26ccab0; no voters:
I20260620 23:01:11.845693 14823 raft_consensus.cc:2781] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:01:11.846169 14823 raft_consensus.cc:686] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0 [term 1 LEADER]: Becoming Leader. State: Replica: f1a60b1cc5154450aba87ca0f26ccab0, State: Running, Role: LEADER
I20260620 23:01:11.846827 14823 consensus_queue.cc:237] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ead0729b1774788972222d3363ce7d8" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 33089 } } peers { permanent_uuid: "d48efd718bcf47f9943ed05c5acb9200" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 40477 } } peers { permanent_uuid: "f1a60b1cc5154450aba87ca0f26ccab0" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 45193 } }
I20260620 23:01:11.852823 14308 catalog_manager.cc:5495] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0 reported cstate change: term changed from 0 to 1, leader changed from <none> to f1a60b1cc5154450aba87ca0f26ccab0 (127.2.237.2). New cstate: current_term: 1 leader_uuid: "f1a60b1cc5154450aba87ca0f26ccab0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ead0729b1774788972222d3363ce7d8" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 33089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d48efd718bcf47f9943ed05c5acb9200" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 40477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1a60b1cc5154450aba87ca0f26ccab0" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 45193 } health_report { overall_health: HEALTHY } } }
I20260620 23:01:11.869249 14816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:01:11.863872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48706 (local address 127.2.237.2:45193)
0620 23:01:11.864140 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0620 23:01:11.864145 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:01:11.864163 (+ 18us) server_negotiation.cc:407] Connection header received
0620 23:01:11.864347 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:01:11.864351 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:01:11.864427 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:01:11.864555 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:01:11.864884 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:01:11.865643 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:01:11.867114 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:01:11.867546 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:01:11.867650 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:01:11.868129 (+ 479us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:01:11.868646 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:01:11.868795 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0620 23:01:11.868853 (+ 58us) server_negotiation.cc:299] Negotiation successful
0620 23:01:11.868923 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260620 23:01:11.878144 14753 raft_consensus.cc:1261] T 330c67dc8e4c4b31ae1ae08b4bcac293 P 4ead0729b1774788972222d3363ce7d8 [term 1 FOLLOWER]: Refusing update from remote peer f1a60b1cc5154450aba87ca0f26ccab0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:01:11.878149 14476 raft_consensus.cc:1261] T 330c67dc8e4c4b31ae1ae08b4bcac293 P d48efd718bcf47f9943ed05c5acb9200 [term 1 FOLLOWER]: Refusing update from remote peer f1a60b1cc5154450aba87ca0f26ccab0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:01:11.879877 14828 consensus_queue.cc:1035] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ead0729b1774788972222d3363ce7d8" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 33089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260620 23:01:11.880275 14823 consensus_queue.cc:1035] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d48efd718bcf47f9943ed05c5acb9200" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 40477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:01:11.894984 14833 mvcc.cc:205] Tried to move back new op lower bound from 7299057548802433024 to 7299057548690608128. Current Snapshot: MvccSnapshot[applied={T|T < 7299057548802433024}]
I20260620 23:01:11.906848 14308 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:01:11.907178 14308 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:01:11.911404 14308 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e4974e83b4143c29283c00beb50561f: Sending DeleteTablet for 3 replicas of tablet 330c67dc8e4c4b31ae1ae08b4bcac293
I20260620 23:01:11.912629 14456 tablet_service.cc:1514] Processing DeleteTablet for tablet 330c67dc8e4c4b31ae1ae08b4bcac293 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:01:11 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:46826
I20260620 23:01:11.912680 14733 tablet_service.cc:1514] Processing DeleteTablet for tablet 330c67dc8e4c4b31ae1ae08b4bcac293 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:01:11 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:47872
I20260620 23:01:11.913883 14841 tablet_replica.cc:330] T 330c67dc8e4c4b31ae1ae08b4bcac293 P d48efd718bcf47f9943ed05c5acb9200: stopping tablet replica
I20260620 23:01:11.913842 14599 tablet_service.cc:1514] Processing DeleteTablet for tablet 330c67dc8e4c4b31ae1ae08b4bcac293 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:01:11 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:48694
I20260620 23:01:11.914443 14842 tablet_replica.cc:330] T 330c67dc8e4c4b31ae1ae08b4bcac293 P 4ead0729b1774788972222d3363ce7d8: stopping tablet replica
I20260620 23:01:11.914935 14842 raft_consensus.cc:2229] T 330c67dc8e4c4b31ae1ae08b4bcac293 P 4ead0729b1774788972222d3363ce7d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:01:11.915482 14842 raft_consensus.cc:2258] T 330c67dc8e4c4b31ae1ae08b4bcac293 P 4ead0729b1774788972222d3363ce7d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:01:11.915515 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 14388
I20260620 23:01:11.916255 14841 raft_consensus.cc:2229] T 330c67dc8e4c4b31ae1ae08b4bcac293 P d48efd718bcf47f9943ed05c5acb9200 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:01:11.916774 14841 raft_consensus.cc:2258] T 330c67dc8e4c4b31ae1ae08b4bcac293 P d48efd718bcf47f9943ed05c5acb9200 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:01:11.920480 14841 ts_tablet_manager.cc:1905] T 330c67dc8e4c4b31ae1ae08b4bcac293 P d48efd718bcf47f9943ed05c5acb9200: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:01:11.923972 14842 ts_tablet_manager.cc:1905] T 330c67dc8e4c4b31ae1ae08b4bcac293 P 4ead0729b1774788972222d3363ce7d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:01:11.928491 14843 tablet_replica.cc:330] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0: stopping tablet replica
I20260620 23:01:11.928990 14843 raft_consensus.cc:2229] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0 [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:01:11.930620 14841 ts_tablet_manager.cc:1918] T 330c67dc8e4c4b31ae1ae08b4bcac293 P d48efd718bcf47f9943ed05c5acb9200: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:01:11.930879 14841 log.cc:1196] T 330c67dc8e4c4b31ae1ae08b4bcac293 P d48efd718bcf47f9943ed05c5acb9200: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-0/wal/wals/330c67dc8e4c4b31ae1ae08b4bcac293
I20260620 23:01:11.932610 14841 ts_tablet_manager.cc:1939] T 330c67dc8e4c4b31ae1ae08b4bcac293 P d48efd718bcf47f9943ed05c5acb9200: Deleting consensus metadata
I20260620 23:01:11.934468 14842 ts_tablet_manager.cc:1918] T 330c67dc8e4c4b31ae1ae08b4bcac293 P 4ead0729b1774788972222d3363ce7d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:01:11.934705 14842 log.cc:1196] T 330c67dc8e4c4b31ae1ae08b4bcac293 P 4ead0729b1774788972222d3363ce7d8: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-2/wal/wals/330c67dc8e4c4b31ae1ae08b4bcac293
I20260620 23:01:11.935230 14842 ts_tablet_manager.cc:1939] T 330c67dc8e4c4b31ae1ae08b4bcac293 P 4ead0729b1774788972222d3363ce7d8: Deleting consensus metadata
I20260620 23:01:11.935246 14294 catalog_manager.cc:4841] TS d48efd718bcf47f9943ed05c5acb9200 (127.2.237.1:40477): tablet 330c67dc8e4c4b31ae1ae08b4bcac293 (table test-table [id=7db6202308f64da6a2b36ed049f8bfac]) successfully deleted
I20260620 23:01:11.936028 14843 raft_consensus.cc:2258] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:01:11.937083 14293 catalog_manager.cc:4841] TS 4ead0729b1774788972222d3363ce7d8 (127.2.237.3:33089): tablet 330c67dc8e4c4b31ae1ae08b4bcac293 (table test-table [id=7db6202308f64da6a2b36ed049f8bfac]) successfully deleted
I20260620 23:01:11.938230 14843 ts_tablet_manager.cc:1905] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0: Deleting tablet data with delete state TABLET_DATA_DELETED
Jun 20 23:01:11 dist-test-slave-f63b krb5kdc[13832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.1@KRBTEST.COM
I20260620 23:01:11.943527 14843 ts_tablet_manager.cc:1918] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:01:11.943709 14843 log.cc:1196] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781996318807807-2996-0/minicluster-data/ts-1/wal/wals/330c67dc8e4c4b31ae1ae08b4bcac293
I20260620 23:01:11.944180 14843 ts_tablet_manager.cc:1939] T 330c67dc8e4c4b31ae1ae08b4bcac293 P f1a60b1cc5154450aba87ca0f26ccab0: Deleting consensus metadata
I20260620 23:01:11.945768 14293 catalog_manager.cc:4841] TS f1a60b1cc5154450aba87ca0f26ccab0 (127.2.237.2:45193): tablet 330c67dc8e4c4b31ae1ae08b4bcac293 (table test-table [id=7db6202308f64da6a2b36ed049f8bfac]) successfully deleted
I20260620 23:01:11.949333 14814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:01:11.916969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46836 (local address 127.2.237.1:40477)
0620 23:01:11.917150 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0620 23:01:11.917158 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:01:11.928066 (+ 10908us) server_negotiation.cc:407] Connection header received
0620 23:01:11.928497 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:01:11.928503 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:01:11.928609 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:01:11.928691 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:01:11.930676 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:01:11.931798 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:01:11.933928 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:01:11.934741 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:01:11.934875 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:01:11.941693 (+ 6818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:01:11.941716 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:01:11.941738 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:01:11.941796 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:01:11.945990 (+ 4194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:01:11.946962 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:01:11.946973 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:01:11.946977 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:01:11.947152 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:01:11.947635 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:01:11.947640 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:01:11.947643 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:01:11.948121 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:01:11.948429 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0620 23:01:11.948690 (+ 261us) server_negotiation.cc:299] Negotiation successful
0620 23:01:11.948921 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260620 23:01:11.950201 14516 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:01:12.041455 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 14388
I20260620 23:01:12.065336 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 14528
Jun 20 23:01:12 dist-test-slave-f63b krb5kdc[13832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.2@KRBTEST.COM
I20260620 23:01:12.084158 14816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:01:12.066136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48718 (local address 127.2.237.2:45193)
0620 23:01:12.066300 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0620 23:01:12.066307 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:01:12.066356 (+ 49us) server_negotiation.cc:407] Connection header received
0620 23:01:12.066547 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:01:12.066551 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:01:12.066694 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:01:12.066779 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:01:12.068527 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:01:12.069538 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:01:12.071627 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:01:12.072071 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:01:12.072184 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:01:12.077656 (+ 5472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:01:12.077673 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:01:12.077693 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:01:12.077744 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:01:12.081115 (+ 3371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:01:12.081902 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:01:12.081910 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:01:12.081913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:01:12.082078 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:01:12.082462 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:01:12.082470 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:01:12.082474 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:01:12.082911 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:01:12.083249 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0620 23:01:12.083547 (+ 298us) server_negotiation.cc:299] Negotiation successful
0620 23:01:12.083774 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260620 23:01:12.084936 14659 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:01:12.200400 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 14528
I20260620 23:01:12.218652 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 14670
Jun 20 23:01:12 dist-test-slave-f63b krb5kdc[13832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.3@KRBTEST.COM
I20260620 23:01:12.234555 14812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:01:12.219612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47886 (local address 127.2.237.3:33089)
0620 23:01:12.219790 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0620 23:01:12.219797 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:01:12.219892 (+ 95us) server_negotiation.cc:407] Connection header received
0620 23:01:12.220142 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:01:12.220149 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:01:12.220220 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:01:12.220320 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:01:12.221633 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:01:12.222338 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:01:12.223819 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:01:12.224201 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:01:12.224330 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:01:12.228794 (+ 4464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:01:12.228813 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:01:12.228831 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:01:12.228873 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:01:12.231703 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:01:12.232444 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:01:12.232452 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:01:12.232454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:01:12.232593 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:01:12.232975 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:01:12.232981 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:01:12.232984 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:01:12.233364 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:01:12.233654 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0620 23:01:12.233969 (+ 315us) server_negotiation.cc:299] Negotiation successful
0620 23:01:12.234201 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260620 23:01:12.235316 14793 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:01:12.324286 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 14670
I20260620 23:01:12.341115 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 14276
I20260620 23:01:12.342335 14338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:01:12.455256 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 14276
I20260620 23:01:12.478165 2996 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260620 23:01:12.814064 2996 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260620 23:01:12.814298 2996 mini_ranger.cc:66] Stopping Ranger...
I20260620 23:01:13.161752 2996 mini_ranger.cc:68] Stopped Ranger
2026-06-20 23:01:13.162 UTC [13862] LOG: received smart shutdown request
2026-06-20 23:01:13.165 UTC [13862] LOG: background worker "logical replication launcher" (PID 13871) exited with exit code 1
2026-06-20 23:01:13.166 UTC [13866] LOG: shutting down
2026-06-20 23:01:13.331 UTC [13862] LOG: database system is shut down
2026-06-20T23:01:13Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (52893 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:01:13 dist-test-slave-f63b krb5kdc[14857](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:01:13 dist-test-slave-f63b krb5kdc[14857](info): set up 2 sockets
Jun 20 23:01:13 dist-test-slave-f63b krb5kdc[14857](info): commencing operation
krb5kdc: starting...
W20260620 23:01:14.586899 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.009s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 23:01:14 dist-test-slave-f63b krb5kdc[14857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:01:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:01:14Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.2.237.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.2.237.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.2.237.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/rangeradmin_127.2.237.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.2.237.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/rangeradmin_127.2.237.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.2.237.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.2.237.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.2.237.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/rangerlookup_127.2.237.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.2.237.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/rangerlookup_127.2.237.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.237.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.2.237.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.237.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/HTTP_127.2.237.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.237.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/HTTP_127.2.237.20@KRBTEST.COM.keytab.
I20260620 23:01:14.921191 2996 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskWPCN6h/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/postgres -l logfile start
2026-06-20 23:01:17.462 UTC [14887] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-20 23:01:17.462 UTC [14887] LOG: listening on IPv4 address "127.2.237.20", port 49451
2026-06-20 23:01:17.465 UTC [14887] LOG: listening on Unix socket "/tmp/.s.PGSQL.49451"
2026-06-20 23:01:17.487 UTC [14890] LOG: database system was shut down at 2026-06-20 23:01:16 UTC
2026-06-20 23:01:17.491 UTC [14887] LOG: database system is ready to accept connections
I20260620 23:01:18.458642 2996 mini_postgres.cc:96] Postgres bound to 49451
2026-06-20 23:01:18.476 UTC [14898] FATAL: database "slave" does not exist
127.2.237.20:49451 - accepting connections
I20260620 23:01:18.476675 2996 mini_ranger.cc:161] Starting Ranger...
I20260620 23:01:18.504410 2996 mini_ranger.cc:84] Created miniranger Postgres user
I20260620 23:01:18.988662 2996 mini_ranger.cc:87] Created ranger Postgres database
I20260620 23:01:18.988777 2996 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ranger-admin
2026-06-20 23:01:19,290 [I] DB FLAVOR :POSTGRES
2026-06-20 23:01:19,290 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 23:01:19,290 [I] Checking connection..
2026-06-20 23:01:19,290 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:49451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-20 23:01:19,566 [I] Checking connection passed.
2026-06-20 23:01:19,566 [I] --------- Verifying version history table ---------
2026-06-20 23:01:19,566 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:49451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 23:01:19,837 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 23:01:19,838 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:49451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 23:01:20,092 [I] Table x_db_version_h does not exist in database ranger
2026-06-20 23:01:20,092 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-20 23:01:20,093 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:49451/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-20 23:01:20.354 UTC [14962] WARNING: there is no transaction in progress
2026-06-20 23:01:20,360 [I] create_dbversion_catalog.sql file imported successfully
2026-06-20 23:01:20,361 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:49451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-20 23:01:20,635 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-20 23:01:20,636 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 23:01:20,636 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:49451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-20 23:01:20,892 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:49451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-20 23:01:21,166 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:49451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-f63b.c.gcp-upstream.internal','N') ;"
2026-06-20 23:01:21,398 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:49451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 23:01:21,670 [I] Table x_portal_user does not exist in database ranger
2026-06-20 23:01:21,671 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:49451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 23:01:21,921 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-20 23:01:21,921 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:49451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 23:01:22,153 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:49451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 23:01:22,402 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-20 23:01:22,403 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:49451/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-20 23:01:23.412 UTC [15070] WARNING: there is no transaction in progress
2026-06-20 23:01:23.430 UTC [15070] WARNING: there is no transaction in progress
2026-06-20 23:01:23.448 UTC [15070] WARNING: there is no transaction in progress
2026-06-20 23:01:23.465 UTC [15070] WARNING: there is no transaction in progress
2026-06-20 23:01:23.482 UTC [15070] WARNING: there is no transaction in progress
2026-06-20 23:01:23.501 UTC [15070] WARNING: there is no transaction in progress
2026-06-20 23:01:23.562 UTC [15070] WARNING: there is no transaction in progress
2026-06-20 23:01:23.569 UTC [15070] WARNING: there is no transaction in progress
2026-06-20 23:01:23.584 UTC [15070] WARNING: there is no transaction in progress
2026-06-20 23:01:23.595 UTC [15070] WARNING: there is no transaction in progress
2026-06-20 23:01:23.606 UTC [15070] WARNING: there is no transaction in progress
2026-06-20 23:01:23.621 UTC [15070] WARNING: there is no transaction in progress
2026-06-20 23:01:23.631 UTC [15070] WARNING: there is no transaction in progress
2026-06-20 23:01:23.638 UTC [15070] WARNING: there is no transaction in progress
2026-06-20 23:01:23.646 UTC [15070] WARNING: there is no transaction in progress
2026-06-20 23:01:23.654 UTC [15070] WARNING: there is no transaction in progress
2026-06-20 23:01:24,340 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-20 23:01:24,340 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:49451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-20 23:01:24,602 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-20 23:01:24,603 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:49451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-20 23:01:24,866 [I] Table x_portal_user already exists in database 'ranger'
2026-06-20 23:01:24,866 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:49451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-20 23:01:25,128 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-20 23:01:25,129 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:49451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-20 23:01:25,364 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:49451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-20 23:01:25,630 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:49451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-f63b.c.gcp-upstream.internal';"
2026-06-20 23:01:25,884 [I] CORE_DB_SCHEMA import status has been updated
2026-06-20 23:01:25,885 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:49451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-20 23:01:26,152 [I] DB_PATCHES have already been applied
I20260620 23:01:26.158932 2996 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ranger-admin:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskWPCN6h/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260620 23:01:26.159056 2996 mini_ranger.cc:193] Using host: 127.2.237.20
I20260620 23:01:26.174120 2996 mini_ranger.cc:227] Ranger admin URL: http://127.2.237.20:54467
Jun 20, 2026 11:01:26 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 11:01:26 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 11:01:26 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ranger-admin/ews/webapp
Jun 20, 2026 11:01:26 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 20, 2026 11:01:26 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 20, 2026 11:01:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ranger-admin/ews/webapp].
Jun 20, 2026 11:01:27 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260620 23:01:27.317039 2996 mini_ranger.cc:160] Time spent starting Ranger: real 8.840s user 0.000s sys 0.041s
Jun 20, 2026 11:01:48 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 20, 2026 11:01:48 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 20, 2026 11:01:48 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 20, 2026 11:01:49 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 20, 2026 11:01:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 20, 2026 11:01:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 20, 2026 11:01:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 20, 2026 11:01:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 20, 2026 11:01:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 20, 2026 11:01:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 20, 2026 11:01:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 20, 2026 11:01:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 20, 2026 11:01:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 20, 2026 11:01:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 20, 2026 11:01:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 20, 2026 11:01:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 20, 2026 11:01:49 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 20, 2026 11:01:49 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260620 23:01:50.598476 2996 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.2.237.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.2.237.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.2.237.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/rangerkms_127.2.237.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.2.237.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/rangerkms_127.2.237.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.237.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.2.237.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.237.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/HTTP_127.2.237.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.237.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/HTTP_127.2.237.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 20 23:01:50 dist-test-slave-f63b krb5kdc[14857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996510, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260620 23:01:50.784049 2996 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260620 23:01:50.805920 2996 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260620 23:01:51.681902 2996 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260620 23:01:51.682029 2996 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ranger-kms
I20260620 23:01:51.682072 2996 mini_ranger_kms.cc:227] Using postgres at 127.2.237.20:49451
2026-06-20 23:01:52,040 [I] DB FLAVOR :POSTGRES
2026-06-20 23:01:52,040 [I] --------- Verifying Ranger DB connection ---------
2026-06-20 23:01:52,040 [I] Checking connection
2026-06-20 23:01:52,040 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:49451/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-20 23:01:52,313 [I] connection success
2026-06-20 23:01:52,314 [I] --------- Verifying Ranger DB tables ---------
2026-06-20 23:01:52,314 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-20 23:01:52,314 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:49451/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-20 23:01:52,593 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-20 23:01:52,593 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-20 23:01:52,593 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-20 23:01:52,594 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.237.20:49451/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-20 23:01:52,942 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260620 23:01:53.538012 2996 mini_ranger_kms.cc:310] Created kms service
I20260620 23:01:53.729581 2996 mini_ranger_kms.cc:326] Created kudu user
I20260620 23:01:53.779150 2996 mini_ranger_kms.cc:343] Created rangerkms user
Jun 20 23:01:53 dist-test-slave-f63b krb5kdc[14857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996510, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.2.237.20@KRBTEST.COM
I20260620 23:01:54.217799 2996 mini_ranger_kms.cc:384] Added ranger policy
I20260620 23:01:54.218128 2996 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ranger-kms:/tmp/dist-test-taskWPCN6h/build/asan/bin/postgresql.jar:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskWPCN6h/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260620 23:01:54.218183 2996 mini_ranger_kms.cc:240] Using host: 127.2.237.20
I20260620 23:01:54.236755 2996 mini_ranger_kms.cc:276] Ranger KMS PID: 15251
I20260620 23:01:54.236944 2996 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.2.237.20:48851
Jun 20, 2026 11:01:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 20, 2026 11:01:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 20, 2026 11:01:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 20, 2026 11:01:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 20, 2026 11:01:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ranger-kms/ews/webapp].
Jun 20, 2026 11:01:55 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260620 23:01:56.578621 2996 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.795s user 0.052s sys 0.321s
I20260620 23:01:56.578886 2996 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260620 23:01:56.579068 2996 mini_ranger_kms.cc:401] 127.2.237.20:48851/kms/v1/keys
Jun 20 23:01:56 dist-test-slave-f63b krb5kdc[14857](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781996516, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.237.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 20 23:01:57 dist-test-slave-f63b krb5kdc[14857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996516, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.237.20@KRBTEST.COM for HTTP/127.2.237.20@KRBTEST.COM
Jun 20 23:01:57 dist-test-slave-f63b krb5kdc[14857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996516, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.237.20@KRBTEST.COM for HTTP/127.2.237.20@KRBTEST.COM
Jun 20 23:02:00 dist-test-slave-f63b krb5kdc[14857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
I20260620 23:02:00.208220 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:46157
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:33613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:46157
--ranger_config_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.237.20:48851/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:02:00.554646 15301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:02:00.554971 15301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:02:00.555154 15301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:02:00.562613 15301 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:02:00.562800 15301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:02:00.562919 15301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:02:00.563027 15301 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:02:00.563122 15301 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:02:00.573807 15301 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:33613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/wal
--ranger_kms_url=127.2.237.20:48851/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:46157
--ranger_config_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:46157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:02:00.575373 15301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:02:00.576828 15301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:02:00.590415 15309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:00.590456 15307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:00.590675 15306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:02:00.591385 15301 server_base.cc:1034] running on GCE node
I20260620 23:02:00.592303 15301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:02:00.593945 15301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:02:00.595219 15301 hybrid_clock.cc:648] HybridClock initialized: now 1781996520595153 us; error 44 us; skew 500 ppm
I20260620 23:02:00.595624 15301 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:02:00 dist-test-slave-f63b krb5kdc[14857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996520, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:02:00.603351 15301 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 23:02:00.605113 15301 webserver.cc:460] Webserver started at http://127.2.237.62:35771/ using document root <none> and password file <none>
I20260620 23:02:00.605878 15301 fs_manager.cc:359] Metadata directory not provided
I20260620 23:02:00.606027 15301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:02:00.606354 15301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 23:02:00 dist-test-slave-f63b krb5kdc[14857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996520, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for HTTP/127.2.237.20@KRBTEST.COM
I20260620 23:02:00.680835 15301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "e6b9b60d17674c028cf50bde5e8e9dba"
format_stamp: "Formatted at 2026-06-20 23:02:00 on dist-test-slave-f63b"
server_key: "8ee400725a5413c8a34d2d84f608ab47"
server_key_iv: "326891a562e2102d1069623dd158433d"
server_key_version: "kuduclusterkey@0"
I20260620 23:02:00.682526 15301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "e6b9b60d17674c028cf50bde5e8e9dba"
format_stamp: "Formatted at 2026-06-20 23:02:00 on dist-test-slave-f63b"
server_key: "8ee400725a5413c8a34d2d84f608ab47"
server_key_iv: "326891a562e2102d1069623dd158433d"
server_key_version: "kuduclusterkey@0"
I20260620 23:02:00.690708 15301 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.003s sys 0.004s
I20260620 23:02:00.716267 15316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:00.718401 15301 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260620 23:02:00.718667 15301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "e6b9b60d17674c028cf50bde5e8e9dba"
format_stamp: "Formatted at 2026-06-20 23:02:00 on dist-test-slave-f63b"
server_key: "8ee400725a5413c8a34d2d84f608ab47"
server_key_iv: "326891a562e2102d1069623dd158433d"
server_key_version: "kuduclusterkey@0"
I20260620 23:02:00.718999 15301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:02:00.738468 15301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:02:00.743152 15301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:02:00.743491 15301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:02:00.763815 15301 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:46157
I20260620 23:02:00.763914 15367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:46157 every 8 connection(s)
I20260620 23:02:00.765318 15301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:02:00.770649 15368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:02:00.773486 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 15301
I20260620 23:02:00.773756 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:02:00.774421 2996 external_mini_cluster.cc:1428] Setting key a4ce2a58707e39e2896707aedc22816d
I20260620 23:02:00.786538 15368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba: Bootstrap starting.
I20260620 23:02:00.791158 15368 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba: Neither blocks nor log segments found. Creating new log.
I20260620 23:02:00.792526 15368 log.cc:824] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba: Log is configured to *not* fsync() on all Append() calls
I20260620 23:02:00.796217 15368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba: No bootstrap required, opened a new log
Jun 20 23:02:00 dist-test-slave-f63b krb5kdc[14857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:02:00.805781 15368 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b9b60d17674c028cf50bde5e8e9dba" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 46157 } }
I20260620 23:02:00.806298 15368 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:02:00.806424 15368 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b9b60d17674c028cf50bde5e8e9dba, State: Initialized, Role: FOLLOWER
I20260620 23:02:00.807238 15368 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b9b60d17674c028cf50bde5e8e9dba" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 46157 } }
I20260620 23:02:00.807533 15368 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:02:00.807687 15368 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:02:00.807883 15368 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:02:00.810467 15368 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b9b60d17674c028cf50bde5e8e9dba" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 46157 } }
I20260620 23:02:00.811132 15368 leader_election.cc:302] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6b9b60d17674c028cf50bde5e8e9dba; no voters:
I20260620 23:02:00.811837 15371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:00.776906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38926 (local address 127.2.237.62:46157)
0620 23:02:00.788022 (+ 11116us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:00.788039 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:00.788596 (+ 557us) server_negotiation.cc:407] Connection header received
0620 23:02:00.791372 (+ 2776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:00.791403 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:00.792026 (+ 623us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:00.792535 (+ 509us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:00.794235 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:00.796286 (+ 2051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:00.797653 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:00.798776 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:00.799004 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:00.802963 (+ 3959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:00.802976 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:00.803004 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:00.803069 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:00.806978 (+ 3909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:00.807937 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:00.807945 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:00.807952 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:00.808163 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:00.808600 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:00.808605 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:00.808609 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:00.809240 (+ 631us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:00.809579 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:00.810050 (+ 471us) server_negotiation.cc:299] Negotiation successful
0620 23:02:00.810502 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10823,"thread_start_us":10536,"threads_started":1}
I20260620 23:02:00.811928 15368 leader_election.cc:288] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:02:00.811998 15373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:02:00.814867 15368 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:02:00.814867 15373 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [term 1 LEADER]: Becoming Leader. State: Replica: e6b9b60d17674c028cf50bde5e8e9dba, State: Running, Role: LEADER
I20260620 23:02:00.815670 15373 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b9b60d17674c028cf50bde5e8e9dba" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 46157 } }
I20260620 23:02:00.818687 15374 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6b9b60d17674c028cf50bde5e8e9dba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b9b60d17674c028cf50bde5e8e9dba" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 46157 } } }
I20260620 23:02:00.819095 15374 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [sys.catalog]: This master's current role is: LEADER
I20260620 23:02:00.818859 15375 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6b9b60d17674c028cf50bde5e8e9dba. Latest consensus state: current_term: 1 leader_uuid: "e6b9b60d17674c028cf50bde5e8e9dba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b9b60d17674c028cf50bde5e8e9dba" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 46157 } } }
I20260620 23:02:00.819228 15375 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [sys.catalog]: This master's current role is: LEADER
I20260620 23:02:00.820765 15383 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:02:00.827950 15383 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:02:00.828809 15368 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260620 23:02:00.854125 15383 catalog_manager.cc:1347] Generated new cluster ID: a7b8107d8a7147cebf3129cd219a2800
I20260620 23:02:00.854277 15383 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:02:00.880740 15383 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:02:00.883033 15383 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:02:00.901850 15383 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba: Generated new TSK 0
I20260620 23:02:00.903225 15383 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.237.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
I20260620 23:02:01.642542 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.237.20:48851/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:0
--local_ip_for_outbound_sockets=127.2.237.1
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:46157
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:33613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 23:02:02 dist-test-slave-f63b krb5kdc[14857](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781996522, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260620 23:02:02.141376 15413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:02:02.141724 15413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:02:02.141908 15413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:02:02.153172 15413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:02:02.153362 15413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:02:02.153573 15413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 23:02:02.171193 15413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:33613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.2.237.20:48851/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:46157
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:02:02.172997 15413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:02:02.175009 15413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:02:02.194130 15423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:02.194187 15424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:02.202282 15426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:02:02.203568 15413 server_base.cc:1034] running on GCE node
I20260620 23:02:02.204448 15413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:02:02.213932 15413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:02:02.215515 15413 hybrid_clock.cc:648] HybridClock initialized: now 1781996522215462 us; error 62 us; skew 500 ppm
I20260620 23:02:02.216058 15413 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:02:02 dist-test-slave-f63b krb5kdc[14857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996522, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:02:02.220532 15413 init.cc:377] Logged in from keytab as kudu/127.2.237.1@KRBTEST.COM (short username kudu)
I20260620 23:02:02.235122 15413 webserver.cc:460] Webserver started at http://127.2.237.1:35921/ using document root <none> and password file <none>
I20260620 23:02:02.236249 15413 fs_manager.cc:359] Metadata directory not provided
I20260620 23:02:02.236426 15413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:02:02.236817 15413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 23:02:02 dist-test-slave-f63b krb5kdc[14857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996522, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for HTTP/127.2.237.20@KRBTEST.COM
I20260620 23:02:02.256788 15413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/data/instance:
uuid: "1cd1f46e108c4a0eaf63842d38744fee"
format_stamp: "Formatted at 2026-06-20 23:02:02 on dist-test-slave-f63b"
server_key: "6810fd03f35e545a1f5d21ef64c279f1"
server_key_iv: "037c3ccf710f08bfbdbf872fbcca96a2"
server_key_version: "kuduclusterkey@0"
I20260620 23:02:02.258265 15413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance:
uuid: "1cd1f46e108c4a0eaf63842d38744fee"
format_stamp: "Formatted at 2026-06-20 23:02:02 on dist-test-slave-f63b"
server_key: "6810fd03f35e545a1f5d21ef64c279f1"
server_key_iv: "037c3ccf710f08bfbdbf872fbcca96a2"
server_key_version: "kuduclusterkey@0"
I20260620 23:02:02.286829 15413 fs_manager.cc:691] Time spent creating directory manager: real 0.028s user 0.013s sys 0.004s
I20260620 23:02:02.325716 15433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:02.328197 15413 fs_manager.cc:725] Time spent opening block manager: real 0.020s user 0.009s sys 0.004s
I20260620 23:02:02.328516 15413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "1cd1f46e108c4a0eaf63842d38744fee"
format_stamp: "Formatted at 2026-06-20 23:02:02 on dist-test-slave-f63b"
server_key: "6810fd03f35e545a1f5d21ef64c279f1"
server_key_iv: "037c3ccf710f08bfbdbf872fbcca96a2"
server_key_version: "kuduclusterkey@0"
I20260620 23:02:02.328926 15413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:02:02.408123 15413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:02:02.414248 15413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:02:02.414743 15413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:02:02.422395 15413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:02:02.431252 15413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:02:02.431468 15413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260620 23:02:02.431691 15413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:02:02.431855 15413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 20 23:02:02 dist-test-slave-f63b krb5kdc[14857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996522, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for HTTP/127.2.237.20@KRBTEST.COM
I20260620 23:02:02.885494 15413 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:35857
I20260620 23:02:02.885562 15545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:35857 every 8 connection(s)
I20260620 23:02:02.887367 15413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 23:02:02.895185 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 15413
I20260620 23:02:02.895467 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance
I20260620 23:02:02.896522 2996 external_mini_cluster.cc:1428] Setting key 423ad729d9747e7035770bc54ee853db
Jun 20 23:02:02 dist-test-slave-f63b krb5kdc[14857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996522, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:02:02.947639 15548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:02.890669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:45267 (local address 127.2.237.62:46157)
0620 23:02:02.899021 (+ 8352us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:02.899028 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:02.903701 (+ 4673us) server_negotiation.cc:407] Connection header received
0620 23:02:02.914802 (+ 11101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:02.914809 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:02.914919 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:02.915012 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:02.917633 (+ 2621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:02.919002 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:02.926200 (+ 7198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:02.926931 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:02.927046 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:02.936368 (+ 9322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:02.936377 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:02.936382 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:02.936446 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:02.940342 (+ 3896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:02.941667 (+ 1325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:02.941674 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:02.941678 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:02.941858 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:02.944956 (+ 3098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:02.944965 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:02.944969 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:02.945423 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:02.945575 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:02.946916 (+ 1341us) server_negotiation.cc:299] Negotiation successful
0620 23:02:02.947202 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8215,"thread_start_us":11924,"threads_started":1}
I20260620 23:02:02.950712 15549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:02.891428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:46157 (local address 127.2.237.1:45267)
0620 23:02:02.903493 (+ 12065us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:02.903558 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:02.905978 (+ 2420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:02.915745 (+ 9767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:02.915769 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:02.916449 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:02:02.917363 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:02.917397 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:02.923365 (+ 5968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:02.923373 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:02.925945 (+ 2572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:02.925982 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:02.927504 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:02.927513 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:02.927847 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:02.928735 (+ 888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:02:02.928782 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:02:02.935920 (+ 7138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:02:02.940642 (+ 4722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:02.940652 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:02.940675 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:02.941496 (+ 821us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:02.942018 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:02.942023 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:02.942026 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:02.942340 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:02.945716 (+ 3376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:02:02.945726 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:02:02.946297 (+ 571us) client_negotiation.cc:769] Sending connection context
0620 23:02:02.948274 (+ 1977us) client_negotiation.cc:240] Negotiation successful
0620 23:02:02.948965 (+ 691us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11686,"thread_start_us":16469,"threads_started":1}
I20260620 23:02:02.957046 15546 heartbeater.cc:347] Connected to a master server at 127.2.237.62:46157
I20260620 23:02:02.957435 15546 heartbeater.cc:464] Registering TS with master...
I20260620 23:02:02.958455 15546 heartbeater.cc:511] Master 127.2.237.62:46157 requested a full tablet report, sending...
I20260620 23:02:02.961548 15333 ts_manager.cc:194] Registered new tserver with Master: 1cd1f46e108c4a0eaf63842d38744fee (127.2.237.1:35857)
I20260620 23:02:02.964052 15333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.1@KRBTEST.COM'} at 127.2.237.1:45267
WARNING: no policy specified for kudu/127.2.237.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
W20260620 23:02:03.012997 2996 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.2.237.2: real 0.113s user 0.006s sys 0.012s
I20260620 23:02:03.014065 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.237.20:48851/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:0
--local_ip_for_outbound_sockets=127.2.237.2
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:46157
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:33613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 23:02:03 dist-test-slave-f63b krb5kdc[14857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996522, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for HTTP/127.2.237.20@KRBTEST.COM
W20260620 23:02:03.512164 15552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:02:03.512476 15552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:02:03.512573 15552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:02:03.520345 15552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:02:03.520478 15552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:02:03.520591 15552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 23:02:03.530817 15552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:33613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.2.237.20:48851/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:46157
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:02:03.532155 15552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:02:03.533455 15552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:02:03.547524 15566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:03.547832 15569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:03.547791 15567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:02:03.548830 15552 server_base.cc:1034] running on GCE node
I20260620 23:02:03.549507 15552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:02:03.550537 15552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:02:03.551824 15552 hybrid_clock.cc:648] HybridClock initialized: now 1781996523551746 us; error 64 us; skew 500 ppm
I20260620 23:02:03.552214 15552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:02:03 dist-test-slave-f63b krb5kdc[14857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996523, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:02:03.555806 15552 init.cc:377] Logged in from keytab as kudu/127.2.237.2@KRBTEST.COM (short username kudu)
I20260620 23:02:03.557499 15552 webserver.cc:460] Webserver started at http://127.2.237.2:41385/ using document root <none> and password file <none>
I20260620 23:02:03.558274 15552 fs_manager.cc:359] Metadata directory not provided
I20260620 23:02:03.558424 15552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:02:03.558835 15552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 23:02:03 dist-test-slave-f63b krb5kdc[14857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996523, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for HTTP/127.2.237.20@KRBTEST.COM
I20260620 23:02:03.573654 15552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/data/instance:
uuid: "2944fc65ae9f438a8ba30b0700840c43"
format_stamp: "Formatted at 2026-06-20 23:02:03 on dist-test-slave-f63b"
server_key: "f3d68f6b4221fd8ed42400f43e29d155"
server_key_iv: "979166ee199d1066a271c10d5b3b497f"
server_key_version: "kuduclusterkey@0"
I20260620 23:02:03.574548 15552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance:
uuid: "2944fc65ae9f438a8ba30b0700840c43"
format_stamp: "Formatted at 2026-06-20 23:02:03 on dist-test-slave-f63b"
server_key: "f3d68f6b4221fd8ed42400f43e29d155"
server_key_iv: "979166ee199d1066a271c10d5b3b497f"
server_key_version: "kuduclusterkey@0"
I20260620 23:02:03.580216 15552 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260620 23:02:03.595086 15576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:03.596860 15552 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260620 23:02:03.597082 15552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "2944fc65ae9f438a8ba30b0700840c43"
format_stamp: "Formatted at 2026-06-20 23:02:03 on dist-test-slave-f63b"
server_key: "f3d68f6b4221fd8ed42400f43e29d155"
server_key_iv: "979166ee199d1066a271c10d5b3b497f"
server_key_version: "kuduclusterkey@0"
I20260620 23:02:03.597343 15552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:02:03.624841 15552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:02:03.629083 15552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:02:03.629427 15552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:02:03.630472 15552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:02:03.631933 15552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:02:03.632046 15552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:03.632197 15552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:02:03.632279 15552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:03.667039 15552 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:46355
I20260620 23:02:03.667135 15688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:46355 every 8 connection(s)
I20260620 23:02:03.668464 15552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 23:02:03.676609 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 15552
I20260620 23:02:03.676805 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance
I20260620 23:02:03.677395 2996 external_mini_cluster.cc:1428] Setting key d9fca541680bd7a4fe0e2ade1403fb7f
Jun 20 23:02:03 dist-test-slave-f63b krb5kdc[14857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996523, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:02:03.690069 15691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:03.671093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:59553 (local address 127.2.237.62:46157)
0620 23:02:03.671609 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:03.671616 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:03.672731 (+ 1115us) server_negotiation.cc:407] Connection header received
0620 23:02:03.674517 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:03.674522 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:03.674650 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:03.674748 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:03.676132 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:03.676991 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:03.679052 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:03.679651 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:03.679750 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:03.684022 (+ 4272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:03.684028 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:03.684031 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:03.684066 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:03.686844 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:03.687662 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:03.687670 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:03.687674 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:03.687801 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:03.688249 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:03.688256 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:03.688259 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:03.688545 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:03.688700 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:03.689434 (+ 734us) server_negotiation.cc:299] Negotiation successful
0620 23:02:03.689676 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":218,"threads_started":1}
I20260620 23:02:03.691207 15692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:03.671832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:46157 (local address 127.2.237.2:59553)
0620 23:02:03.672553 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:03.672598 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:03.674260 (+ 1662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:03.674959 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:03.674967 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:03.675386 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:02:03.675981 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:03.675997 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:03.677134 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:03.677140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:03.678917 (+ 1777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:03.678929 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:03.679917 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:03.679925 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:03.680269 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:03.681018 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:02:03.681061 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:02:03.683736 (+ 2675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:02:03.687016 (+ 3280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:03.687025 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:03.687045 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:03.687530 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:03.687924 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:03.687930 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:03.687937 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:03.688138 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:03.688676 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:02:03.688684 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:02:03.689053 (+ 369us) client_negotiation.cc:769] Sending connection context
0620 23:02:03.689355 (+ 302us) client_negotiation.cc:240] Negotiation successful
0620 23:02:03.689908 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":294,"threads_started":1}
I20260620 23:02:03.692862 15689 heartbeater.cc:347] Connected to a master server at 127.2.237.62:46157
I20260620 23:02:03.693212 15689 heartbeater.cc:464] Registering TS with master...
I20260620 23:02:03.694145 15689 heartbeater.cc:511] Master 127.2.237.62:46157 requested a full tablet report, sending...
I20260620 23:02:03.695948 15333 ts_manager.cc:194] Registered new tserver with Master: 2944fc65ae9f438a8ba30b0700840c43 (127.2.237.2:46355)
I20260620 23:02:03.697283 15333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.2@KRBTEST.COM'} at 127.2.237.2:59553
WARNING: no policy specified for kudu/127.2.237.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
I20260620 23:02:03.731853 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.237.20:48851/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:0
--local_ip_for_outbound_sockets=127.2.237.3
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:46157
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:33613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:02:03.968807 15546 heartbeater.cc:503] Master 127.2.237.62:46157 was elected leader, sending a full tablet report...
W20260620 23:02:04.075304 15695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:02:04.075614 15695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:02:04.075773 15695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:02:04.083397 15695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:02:04.083540 15695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:02:04.083727 15695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 23:02:04.093837 15695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:33613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.2.237.20:48851/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:46157
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:02:04.095294 15695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:02:04.096637 15695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:02:04.110942 15701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:04.110903 15703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:04.110898 15700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:02:04.111616 15695 server_base.cc:1034] running on GCE node
I20260620 23:02:04.112329 15695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:02:04.113354 15695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:02:04.114645 15695 hybrid_clock.cc:648] HybridClock initialized: now 1781996524114533 us; error 64 us; skew 500 ppm
I20260620 23:02:04.115020 15695 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:02:04 dist-test-slave-f63b krb5kdc[14857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996524, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:02:04.118530 15695 init.cc:377] Logged in from keytab as kudu/127.2.237.3@KRBTEST.COM (short username kudu)
I20260620 23:02:04.120203 15695 webserver.cc:460] Webserver started at http://127.2.237.3:38559/ using document root <none> and password file <none>
I20260620 23:02:04.120949 15695 fs_manager.cc:359] Metadata directory not provided
I20260620 23:02:04.121097 15695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:02:04.121421 15695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 20 23:02:04 dist-test-slave-f63b krb5kdc[14857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996524, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for HTTP/127.2.237.20@KRBTEST.COM
I20260620 23:02:04.135944 15695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/data/instance:
uuid: "6cdcbf7b54604d449e5240f55427ff72"
format_stamp: "Formatted at 2026-06-20 23:02:04 on dist-test-slave-f63b"
server_key: "a2fd9cfe4183a09968c3ccc73c8d6932"
server_key_iv: "2f24197d327bc692524d151fa96fac9f"
server_key_version: "kuduclusterkey@0"
I20260620 23:02:04.136925 15695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance:
uuid: "6cdcbf7b54604d449e5240f55427ff72"
format_stamp: "Formatted at 2026-06-20 23:02:04 on dist-test-slave-f63b"
server_key: "a2fd9cfe4183a09968c3ccc73c8d6932"
server_key_iv: "2f24197d327bc692524d151fa96fac9f"
server_key_version: "kuduclusterkey@0"
I20260620 23:02:04.142906 15695 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260620 23:02:04.159866 15710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:04.161435 15695 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260620 23:02:04.161741 15695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "6cdcbf7b54604d449e5240f55427ff72"
format_stamp: "Formatted at 2026-06-20 23:02:04 on dist-test-slave-f63b"
server_key: "a2fd9cfe4183a09968c3ccc73c8d6932"
server_key_iv: "2f24197d327bc692524d151fa96fac9f"
server_key_version: "kuduclusterkey@0"
I20260620 23:02:04.161996 15695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:02:04.193713 15695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:02:04.197722 15695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:02:04.198047 15695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:02:04.199126 15695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:02:04.200541 15695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:02:04.200656 15695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:04.200814 15695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:02:04.200887 15695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:04.233704 15695 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:36169
I20260620 23:02:04.233742 15822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:36169 every 8 connection(s)
I20260620 23:02:04.235061 15695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 23:02:04.236301 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 15695
I20260620 23:02:04.236505 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance
I20260620 23:02:04.237138 2996 external_mini_cluster.cc:1428] Setting key 88d7b6d46ba98ab342e9e6ed16a74318
Jun 20 23:02:04 dist-test-slave-f63b krb5kdc[14857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996524, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:02:04.255924 15825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:04.238080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:59175 (local address 127.2.237.62:46157)
0620 23:02:04.238700 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:04.238706 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:04.239947 (+ 1241us) server_negotiation.cc:407] Connection header received
0620 23:02:04.241887 (+ 1940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:04.241891 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:04.241970 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:04.242060 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:04.243591 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:04.244378 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:04.245892 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:04.246314 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:04.246412 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:04.249928 (+ 3516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:04.249933 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:04.249936 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:04.249973 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:04.252755 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:04.253546 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:04.253556 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:04.253559 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:04.253688 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:04.254133 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:04.254137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:04.254140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:04.254402 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:04.254574 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:04.255300 (+ 726us) server_negotiation.cc:299] Negotiation successful
0620 23:02:04.255533 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":300,"threads_started":1}
I20260620 23:02:04.256968 15826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:04.238810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:46157 (local address 127.2.237.3:59175)
0620 23:02:04.239759 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:04.239819 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:04.241637 (+ 1818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:04.242275 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:04.242283 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:04.242792 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:02:04.243393 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:04.243411 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:04.244501 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:04.244505 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:04.245748 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:04.245766 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:04.246500 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:04.246504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:04.246771 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:04.247588 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:02:04.247611 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:02:04.249662 (+ 2051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:02:04.252916 (+ 3254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:04.252926 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:04.252947 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:04.253413 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:04.253801 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:04.253806 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:04.253810 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:04.254007 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:04.254539 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:02:04.254547 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:02:04.254934 (+ 387us) client_negotiation.cc:769] Sending connection context
0620 23:02:04.255209 (+ 275us) client_negotiation.cc:240] Negotiation successful
0620 23:02:04.255770 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":401,"threads_started":1}
I20260620 23:02:04.258435 15823 heartbeater.cc:347] Connected to a master server at 127.2.237.62:46157
I20260620 23:02:04.258759 15823 heartbeater.cc:464] Registering TS with master...
I20260620 23:02:04.259483 15823 heartbeater.cc:511] Master 127.2.237.62:46157 requested a full tablet report, sending...
I20260620 23:02:04.260941 15333 ts_manager.cc:194] Registered new tserver with Master: 6cdcbf7b54604d449e5240f55427ff72 (127.2.237.3:36169)
I20260620 23:02:04.261775 15333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.3@KRBTEST.COM'} at 127.2.237.3:59175
I20260620 23:02:04.265331 2996 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:02:04.267103 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 15413
Jun 20 23:02:04 dist-test-slave-f63b krb5kdc[14857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.1@KRBTEST.COM
I20260620 23:02:04.284350 15828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:04.268101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39114 (local address 127.2.237.1:35857)
0620 23:02:04.268749 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:04.268757 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:04.268789 (+ 32us) server_negotiation.cc:407] Connection header received
0620 23:02:04.268850 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:04.268857 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:04.269064 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:04.269243 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:04.270939 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:04.271978 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:04.273494 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:04.274075 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:04.274219 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:04.278673 (+ 4454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:04.278688 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:04.278705 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:04.278742 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:04.281442 (+ 2700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:04.282249 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:04.282257 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:04.282259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:04.282414 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:04.282925 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:04.282929 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:04.282931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:04.283311 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:04.283573 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:04.283841 (+ 268us) server_negotiation.cc:299] Negotiation successful
0620 23:02:04.284064 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":297,"threads_started":1}
I20260620 23:02:04.285742 15541 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:02:04.369095 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 15413
I20260620 23:02:04.385100 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 15552
Jun 20 23:02:04 dist-test-slave-f63b krb5kdc[14857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.2@KRBTEST.COM
I20260620 23:02:04.406456 15830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:04.385995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49806 (local address 127.2.237.2:46355)
0620 23:02:04.386487 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:04.386494 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:04.386519 (+ 25us) server_negotiation.cc:407] Connection header received
0620 23:02:04.386637 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:04.386644 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:04.386889 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:04.387022 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:04.388291 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:04.389261 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:04.390635 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:04.391215 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:04.391315 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:04.400174 (+ 8859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:04.400191 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:04.400207 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:04.400249 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:04.403399 (+ 3150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:04.404206 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:04.404215 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:04.404217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:04.404357 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:04.404772 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:04.404778 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:04.404782 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:04.405153 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:04.405473 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:04.405807 (+ 334us) server_negotiation.cc:299] Negotiation successful
0620 23:02:04.406091 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":262,"threads_started":1}
I20260620 23:02:04.407831 15684 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:02:04.491129 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 15552
I20260620 23:02:04.507917 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 15695
Jun 20 23:02:04 dist-test-slave-f63b krb5kdc[14857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.3@KRBTEST.COM
I20260620 23:02:04.525103 15832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:04.508861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49946 (local address 127.2.237.3:36169)
0620 23:02:04.509484 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:04.509491 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:04.509519 (+ 28us) server_negotiation.cc:407] Connection header received
0620 23:02:04.509578 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:04.509584 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:04.509828 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:04.510013 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:04.511434 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:04.512445 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:04.513755 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:04.514355 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:04.514502 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:04.519127 (+ 4625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:04.519139 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:04.519154 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:04.519194 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:04.522016 (+ 2822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:04.522817 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:04.522824 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:04.522827 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:04.522968 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:04.523374 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:04.523386 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:04.523389 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:04.523731 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:04.524085 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:04.524373 (+ 288us) server_negotiation.cc:299] Negotiation successful
0620 23:02:04.524682 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":313,"threads_started":1}
I20260620 23:02:04.526506 15818 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:02:04.615767 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 15695
I20260620 23:02:04.631883 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 15301
I20260620 23:02:04.633200 15363 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:02:04.742983 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 15301
I20260620 23:02:04.767695 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:46157
--webserver_interface=127.2.237.62
--webserver_port=35771
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:33613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:46157
--ranger_config_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.237.20:48851/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.237.20:48851/kms with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:02:05.099992 15835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:02:05.100373 15835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:02:05.100538 15835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:02:05.107972 15835 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:02:05.108141 15835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:02:05.108259 15835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:02:05.108362 15835 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:02:05.108448 15835 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:02:05.119992 15835 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:33613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.237.20:48851/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:46157
--ranger_config_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:46157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=35771
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:02:05.121419 15835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:02:05.122869 15835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:02:05.135115 15840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:05.135010 15843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:05.135089 15841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:02:05.136025 15835 server_base.cc:1034] running on GCE node
I20260620 23:02:05.136754 15835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:02:05.138155 15835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:02:05.139358 15835 hybrid_clock.cc:648] HybridClock initialized: now 1781996525139321 us; error 27 us; skew 500 ppm
I20260620 23:02:05.139736 15835 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:02:05 dist-test-slave-f63b krb5kdc[14857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996525, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:02:05.142918 15835 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 23:02:05.144529 15835 webserver.cc:460] Webserver started at http://127.2.237.62:35771/ using document root <none> and password file <none>
I20260620 23:02:05.145287 15835 fs_manager.cc:359] Metadata directory not provided
I20260620 23:02:05.145439 15835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 23:02:05 dist-test-slave-f63b krb5kdc[14857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996525, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for HTTP/127.2.237.20@KRBTEST.COM
I20260620 23:02:05.230794 15835 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260620 23:02:05.233948 15851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:05.235430 15835 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 23:02:05.235648 15835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "e6b9b60d17674c028cf50bde5e8e9dba"
format_stamp: "Formatted at 2026-06-20 23:02:00 on dist-test-slave-f63b"
server_key: "8ee400725a5413c8a34d2d84f608ab47"
server_key_iv: "326891a562e2102d1069623dd158433d"
server_key_version: "kuduclusterkey@0"
I20260620 23:02:05.236658 15835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:02:05.252130 15835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:02:05.256139 15835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:02:05.256457 15835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:02:05.276103 15835 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:46157
I20260620 23:02:05.276124 15902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:46157 every 8 connection(s)
I20260620 23:02:05.277740 15835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:02:05.282531 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 15835
I20260620 23:02:05.283607 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.237.20:48851/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:35857
--local_ip_for_outbound_sockets=127.2.237.1
--tserver_master_addrs=127.2.237.62:46157
--webserver_port=35921
--webserver_interface=127.2.237.1
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:33613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.237.20:48851/kms with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:02:05.285890 15903 sys_catalog.cc:263] Verifying existing consensus state
I20260620 23:02:05.291473 15903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba: Bootstrap starting.
I20260620 23:02:05.311007 15903 log.cc:824] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba: Log is configured to *not* fsync() on all Append() calls
I20260620 23:02:05.320240 15903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 23:02:05.320838 15903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba: Bootstrap complete.
I20260620 23:02:05.330447 15903 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b9b60d17674c028cf50bde5e8e9dba" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 46157 } }
I20260620 23:02:05.331416 15903 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b9b60d17674c028cf50bde5e8e9dba, State: Initialized, Role: FOLLOWER
I20260620 23:02:05.332086 15903 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b9b60d17674c028cf50bde5e8e9dba" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 46157 } }
I20260620 23:02:05.332326 15903 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:02:05.332466 15903 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:02:05.332653 15903 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [term 1 FOLLOWER]: Advancing to term 2
I20260620 23:02:05.336541 15903 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b9b60d17674c028cf50bde5e8e9dba" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 46157 } }
I20260620 23:02:05.337066 15903 leader_election.cc:302] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6b9b60d17674c028cf50bde5e8e9dba; no voters:
I20260620 23:02:05.337723 15903 leader_election.cc:288] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 23:02:05.337909 15907 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [term 2 FOLLOWER]: Leader election won for term 2
I20260620 23:02:05.339005 15907 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [term 2 LEADER]: Becoming Leader. State: Replica: e6b9b60d17674c028cf50bde5e8e9dba, State: Running, Role: LEADER
I20260620 23:02:05.339663 15907 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b9b60d17674c028cf50bde5e8e9dba" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 46157 } }
I20260620 23:02:05.340109 15903 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:02:05.348212 15909 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6b9b60d17674c028cf50bde5e8e9dba. Latest consensus state: current_term: 2 leader_uuid: "e6b9b60d17674c028cf50bde5e8e9dba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b9b60d17674c028cf50bde5e8e9dba" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 46157 } } }
I20260620 23:02:05.348717 15909 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [sys.catalog]: This master's current role is: LEADER
I20260620 23:02:05.349346 15908 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e6b9b60d17674c028cf50bde5e8e9dba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b9b60d17674c028cf50bde5e8e9dba" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 46157 } } }
I20260620 23:02:05.349664 15908 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba [sys.catalog]: This master's current role is: LEADER
I20260620 23:02:05.353274 15920 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:02:05.359160 15920 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:02:05.359982 15903 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260620 23:02:05.360791 15920 catalog_manager.cc:1259] Loaded cluster ID: a7b8107d8a7147cebf3129cd219a2800
I20260620 23:02:05.360953 15920 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:02:05.366178 15920 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:02:05.368410 15920 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba: Loaded TSK: 0
I20260620 23:02:05.370636 15920 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 23:02:05.765905 15905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:02:05.766325 15905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:02:05.766549 15905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:02:05.778509 15905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:02:05.778760 15905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:02:05.779014 15905 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 23:02:05.795399 15905 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:33613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.237.20:48851/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:35857
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=35921
--tserver_master_addrs=127.2.237.62:46157
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:02:05.797480 15905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:02:05.799342 15905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:02:05.823433 15941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:05.823457 15942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:05.832880 15944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:02:05.833482 15905 server_base.cc:1034] running on GCE node
I20260620 23:02:05.834553 15905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:02:05.836050 15905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:02:05.837414 15905 hybrid_clock.cc:648] HybridClock initialized: now 1781996525837344 us; error 65 us; skew 500 ppm
I20260620 23:02:05.837837 15905 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:02:05 dist-test-slave-f63b krb5kdc[14857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996525, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:02:05.854948 15905 init.cc:377] Logged in from keytab as kudu/127.2.237.1@KRBTEST.COM (short username kudu)
I20260620 23:02:05.863355 15905 webserver.cc:460] Webserver started at http://127.2.237.1:35921/ using document root <none> and password file <none>
I20260620 23:02:05.864588 15905 fs_manager.cc:359] Metadata directory not provided
I20260620 23:02:05.864809 15905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 23:02:05 dist-test-slave-f63b krb5kdc[14857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996525, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for HTTP/127.2.237.20@KRBTEST.COM
I20260620 23:02:05.918952 15905 fs_manager.cc:709] Time spent opening directory manager: real 0.020s user 0.005s sys 0.007s
I20260620 23:02:05.944978 15953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:05.953724 15905 fs_manager.cc:725] Time spent opening block manager: real 0.032s user 0.014s sys 0.012s
I20260620 23:02:05.954005 15905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "1cd1f46e108c4a0eaf63842d38744fee"
format_stamp: "Formatted at 2026-06-20 23:02:02 on dist-test-slave-f63b"
server_key: "6810fd03f35e545a1f5d21ef64c279f1"
server_key_iv: "037c3ccf710f08bfbdbf872fbcca96a2"
server_key_version: "kuduclusterkey@0"
I20260620 23:02:05.955909 15905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:02:06.015542 15905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:02:06.020213 15905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:02:06.020519 15905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:02:06.021638 15905 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:02:06.023366 15905 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:02:06.023521 15905 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260620 23:02:06.023658 15905 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:02:06.023756 15905 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:06.102769 15905 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:35857
I20260620 23:02:06.102854 16065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:35857 every 8 connection(s)
I20260620 23:02:06.104799 15905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 23:02:06.110373 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 15905
I20260620 23:02:06.111235 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.237.20:48851/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:46355
--local_ip_for_outbound_sockets=127.2.237.2
--tserver_master_addrs=127.2.237.62:46157
--webserver_port=41385
--webserver_interface=127.2.237.2
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:33613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.237.20:48851/kms with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 23:02:06 dist-test-slave-f63b krb5kdc[14857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996525, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:02:06.148432 16069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:06.108853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:46157 (local address 127.2.237.1:50111)
0620 23:02:06.115461 (+ 6608us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:06.115526 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:06.118170 (+ 2644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:06.121400 (+ 3230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:06.121413 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:06.122094 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:02:06.123105 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:06.123145 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:06.126027 (+ 2882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:06.126033 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:06.128574 (+ 2541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:06.128610 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:06.131002 (+ 2392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:06.131012 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:06.131397 (+ 385us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:06.133429 (+ 2032us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:02:06.133473 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:02:06.136770 (+ 3297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:02:06.142123 (+ 5353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:06.142146 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:06.142167 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:06.142848 (+ 681us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:06.143454 (+ 606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:06.143462 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:06.143467 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:06.143806 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:06.145144 (+ 1338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:02:06.145152 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:02:06.145596 (+ 444us) client_negotiation.cc:769] Sending connection context
0620 23:02:06.145981 (+ 385us) client_negotiation.cc:240] Negotiation successful
0620 23:02:06.146741 (+ 760us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6240,"thread_start_us":5972,"threads_started":1}
I20260620 23:02:06.148432 16068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:06.108801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:50111 (local address 127.2.237.62:46157)
0620 23:02:06.109833 (+ 1032us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:06.109864 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:06.115747 (+ 5883us) server_negotiation.cc:407] Connection header received
0620 23:02:06.119252 (+ 3505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:06.119280 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:06.119981 (+ 701us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:06.120482 (+ 501us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:06.123398 (+ 2916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:06.125788 (+ 2390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:06.128856 (+ 3068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:06.130149 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:06.130367 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:06.137262 (+ 6895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:06.137281 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:06.137309 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:06.137394 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:06.141902 (+ 4508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:06.143064 (+ 1162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:06.143074 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:06.143081 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:06.143294 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:06.143968 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:06.143975 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:06.143977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:06.144773 (+ 796us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:06.145060 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:06.146287 (+ 1227us) server_negotiation.cc:299] Negotiation successful
0620 23:02:06.146784 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":345,"threads_started":1}
I20260620 23:02:06.153394 16066 heartbeater.cc:347] Connected to a master server at 127.2.237.62:46157
I20260620 23:02:06.153734 16066 heartbeater.cc:464] Registering TS with master...
I20260620 23:02:06.154840 16066 heartbeater.cc:511] Master 127.2.237.62:46157 requested a full tablet report, sending...
W20260620 23:02:06.159433 16066 heartbeater.cc:644] Failed to heartbeat to 127.2.237.62:46157 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 23:02:06.159772 16066 heartbeater.cc:464] Registering TS with master...
I20260620 23:02:06.160233 16066 heartbeater.cc:511] Master 127.2.237.62:46157 requested a full tablet report, sending...
I20260620 23:02:06.162204 16066 heartbeater.cc:464] Registering TS with master...
I20260620 23:02:06.162649 16066 heartbeater.cc:511] Master 127.2.237.62:46157 requested a full tablet report, sending...
W20260620 23:02:06.164377 16066 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 23:02:06.606048 16070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:02:06.606647 16070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:02:06.606863 16070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:02:06.619210 16070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:02:06.619451 16070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:02:06.619684 16070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 23:02:06.636133 16070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:33613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.237.20:48851/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:46355
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=41385
--tserver_master_addrs=127.2.237.62:46157
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:02:06.638270 16070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:02:06.640280 16070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:02:06.679168 16087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
Jun 20 23:02:06 dist-test-slave-f63b krb5kdc[14857](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781996526, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260620 23:02:06.679339 16086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:06.681965 16090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:02:06.682919 16070 server_base.cc:1034] running on GCE node
I20260620 23:02:06.684031 16070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:02:06.698925 16070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:02:06.710701 16070 hybrid_clock.cc:648] HybridClock initialized: now 1781996526710605 us; error 70 us; skew 500 ppm
I20260620 23:02:06.711336 16070 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:02:06 dist-test-slave-f63b krb5kdc[14857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996526, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:02:06.723914 16070 init.cc:377] Logged in from keytab as kudu/127.2.237.2@KRBTEST.COM (short username kudu)
I20260620 23:02:06.743371 16070 webserver.cc:460] Webserver started at http://127.2.237.2:41385/ using document root <none> and password file <none>
I20260620 23:02:06.744580 16070 fs_manager.cc:359] Metadata directory not provided
I20260620 23:02:06.744879 16070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 23:02:06 dist-test-slave-f63b krb5kdc[14857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996526, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for HTTP/127.2.237.20@KRBTEST.COM
I20260620 23:02:06.788955 16070 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.004s sys 0.000s
I20260620 23:02:06.799491 16097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:06.801836 16070 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.004s sys 0.004s
I20260620 23:02:06.802081 16070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "2944fc65ae9f438a8ba30b0700840c43"
format_stamp: "Formatted at 2026-06-20 23:02:03 on dist-test-slave-f63b"
server_key: "f3d68f6b4221fd8ed42400f43e29d155"
server_key_iv: "979166ee199d1066a271c10d5b3b497f"
server_key_version: "kuduclusterkey@0"
I20260620 23:02:06.804059 16070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:02:06.855687 16070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:02:06.862704 16070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:02:06.863181 16070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:02:06.867060 16070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:02:06.868960 16070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:02:06.869057 16070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:06.869199 16070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:02:06.869273 16070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:06.932133 16070 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:46355
I20260620 23:02:06.934213 16070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 23:02:06.932190 16210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:46355 every 8 connection(s)
I20260620 23:02:06.936827 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 16070
I20260620 23:02:06.937852 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.237.20:48851/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:36169
--local_ip_for_outbound_sockets=127.2.237.3
--tserver_master_addrs=127.2.237.62:46157
--webserver_port=38559
--webserver_interface=127.2.237.3
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:33613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.237.20:48851/kms with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 23:02:06 dist-test-slave-f63b krb5kdc[14857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996526, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:02:06.973132 16213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:06.941662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:46157 (local address 127.2.237.2:60475)
0620 23:02:06.942675 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:06.942746 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:06.945276 (+ 2530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:06.946746 (+ 1470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:06.946758 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:06.947454 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:02:06.948509 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:06.948542 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:06.950204 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:06.950210 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:06.952394 (+ 2184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:06.952415 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:06.953807 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:06.953817 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:06.954187 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:06.959509 (+ 5322us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:02:06.959553 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:02:06.963098 (+ 3545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:02:06.967591 (+ 4493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:06.967602 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:06.967625 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:06.968350 (+ 725us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:06.968880 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:06.968885 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:06.968888 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:06.969169 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:06.969969 (+ 800us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:02:06.969978 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:02:06.970498 (+ 520us) client_negotiation.cc:769] Sending connection context
0620 23:02:06.970860 (+ 362us) client_negotiation.cc:240] Negotiation successful
0620 23:02:06.971512 (+ 652us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":645,"threads_started":1}
I20260620 23:02:06.974438 16214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:06.942312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:60475 (local address 127.2.237.62:46157)
0620 23:02:06.946054 (+ 3742us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:06.946061 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:06.946081 (+ 20us) server_negotiation.cc:407] Connection header received
0620 23:02:06.946143 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:06.946148 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:06.946230 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:06.946361 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:06.948788 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:06.949961 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:06.952624 (+ 2663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:06.953290 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:06.953413 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:06.963481 (+ 10068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:06.963490 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:06.963498 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:06.963553 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:06.967284 (+ 3731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:06.968544 (+ 1260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:06.968549 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:06.968553 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:06.968723 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:06.969337 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:06.969342 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:06.969345 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:06.969730 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:06.969891 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:06.973728 (+ 3837us) server_negotiation.cc:299] Negotiation successful
0620 23:02:06.974024 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3597,"thread_start_us":3478,"threads_started":1}
I20260620 23:02:06.976934 16211 heartbeater.cc:347] Connected to a master server at 127.2.237.62:46157
I20260620 23:02:06.977309 16211 heartbeater.cc:464] Registering TS with master...
I20260620 23:02:06.978245 16211 heartbeater.cc:511] Master 127.2.237.62:46157 requested a full tablet report, sending...
I20260620 23:02:06.980624 15868 ts_manager.cc:194] Registered new tserver with Master: 2944fc65ae9f438a8ba30b0700840c43 (127.2.237.2:46355)
I20260620 23:02:06.982939 15868 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.2@KRBTEST.COM'} at 127.2.237.2:60475
I20260620 23:02:07.164567 16066 heartbeater.cc:464] Registering TS with master...
I20260620 23:02:07.165041 16066 heartbeater.cc:511] Master 127.2.237.62:46157 requested a full tablet report, sending...
I20260620 23:02:07.166639 15868 ts_manager.cc:194] Registered new tserver with Master: 1cd1f46e108c4a0eaf63842d38744fee (127.2.237.1:35857)
I20260620 23:02:07.167894 15868 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.1@KRBTEST.COM'} at 127.2.237.1:50111
Jun 20 23:02:07 dist-test-slave-f63b krb5kdc[14857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996526, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for HTTP/127.2.237.20@KRBTEST.COM
W20260620 23:02:07.426196 16215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:02:07.426687 16215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:02:07.426891 16215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:02:07.438287 16215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:02:07.438495 16215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:02:07.438719 16215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 23:02:07.453140 16215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:33613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.237.20:48851/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:36169
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=38559
--tserver_master_addrs=127.2.237.62:46157
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:02:07.455075 16215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:02:07.456899 16215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:02:07.473876 16220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:07.474431 16221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:07.492193 16223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:02:07.496438 16215 server_base.cc:1034] running on GCE node
I20260620 23:02:07.497295 16215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:02:07.498871 16215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:02:07.500195 16215 hybrid_clock.cc:648] HybridClock initialized: now 1781996527500131 us; error 53 us; skew 500 ppm
I20260620 23:02:07.500586 16215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:02:07 dist-test-slave-f63b krb5kdc[14857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996527, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:02:07.504859 16215 init.cc:377] Logged in from keytab as kudu/127.2.237.3@KRBTEST.COM (short username kudu)
I20260620 23:02:07.509335 16215 webserver.cc:460] Webserver started at http://127.2.237.3:38559/ using document root <none> and password file <none>
I20260620 23:02:07.510386 16215 fs_manager.cc:359] Metadata directory not provided
I20260620 23:02:07.510645 16215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 20 23:02:07 dist-test-slave-f63b krb5kdc[14857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996527, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for HTTP/127.2.237.20@KRBTEST.COM
I20260620 23:02:07.537122 16215 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260620 23:02:07.543490 16231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:07.545701 16215 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.003s
I20260620 23:02:07.546018 16215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "6cdcbf7b54604d449e5240f55427ff72"
format_stamp: "Formatted at 2026-06-20 23:02:04 on dist-test-slave-f63b"
server_key: "a2fd9cfe4183a09968c3ccc73c8d6932"
server_key_iv: "2f24197d327bc692524d151fa96fac9f"
server_key_version: "kuduclusterkey@0"
I20260620 23:02:07.547663 16215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Jun 20 23:02:07 dist-test-slave-f63b krb5kdc[14857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996526, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for HTTP/127.2.237.20@KRBTEST.COM
I20260620 23:02:07.601033 16215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:02:07.606791 16215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:02:07.607218 16215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:02:07.636439 16215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:02:07.644861 16215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:02:07.645085 16215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260620 23:02:07.645262 16215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:02:07.645395 16215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:07.806906 16215 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:36169
I20260620 23:02:07.806964 16344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:36169 every 8 connection(s)
I20260620 23:02:07.809095 16215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 23:02:07.818500 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 16215
Jun 20 23:02:07 dist-test-slave-f63b krb5kdc[14857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996527, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:02:07.851089 16350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:07.819578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53412 (local address 127.2.237.62:46157)
0620 23:02:07.831350 (+ 11772us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:07.831355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:07.831374 (+ 19us) server_negotiation.cc:407] Connection header received
0620 23:02:07.831443 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:07.831448 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:07.831560 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:07.831691 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:07.834373 (+ 2682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:07.835550 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:07.838608 (+ 3058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:07.839250 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:07.839389 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:07.843251 (+ 3862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:07.843257 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:07.843262 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:07.843315 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:07.847771 (+ 4456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:07.848725 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:07.848732 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:07.848735 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:07.848872 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:07.849498 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:07.849502 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:07.849504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:07.849799 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:07.850119 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:07.850461 (+ 342us) server_negotiation.cc:299] Negotiation successful
0620 23:02:07.850732 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11655,"thread_start_us":11081,"threads_started":1}
I20260620 23:02:07.854357 16351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:07.815369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:55083 (local address 127.2.237.62:46157)
0620 23:02:07.830958 (+ 15589us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:07.830964 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:07.832601 (+ 1637us) server_negotiation.cc:407] Connection header received
0620 23:02:07.835007 (+ 2406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:07.835013 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:07.835120 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:07.835222 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:07.837592 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:07.838808 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:07.841828 (+ 3020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:07.842886 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:07.843017 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:07.847443 (+ 4426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:07.847451 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:07.847456 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:07.847504 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:07.850545 (+ 3041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:07.851617 (+ 1072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:07.851625 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:07.851631 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:07.851764 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:07.852428 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:07.852432 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:07.852435 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:07.852732 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:07.852844 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:07.853789 (+ 945us) server_negotiation.cc:299] Negotiation successful
0620 23:02:07.854087 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15459,"thread_start_us":15707,"threads_started":1}
I20260620 23:02:07.855470 16349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:07.814742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:46157 (local address 127.2.237.3:55083)
0620 23:02:07.832409 (+ 17667us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:07.832453 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:07.834423 (+ 1970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:07.835756 (+ 1333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:07.835770 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:07.836441 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:02:07.837315 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:07.837345 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:07.839005 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:07.839011 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:07.841633 (+ 2622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:07.841656 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:07.843248 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:07.843256 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:07.843570 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:07.844368 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:02:07.844404 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:02:07.847077 (+ 2673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:02:07.850763 (+ 3686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:07.850772 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:07.850791 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:07.851485 (+ 694us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:07.851994 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:07.852001 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:07.852006 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:07.852285 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:07.852858 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:02:07.852868 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:02:07.853353 (+ 485us) client_negotiation.cc:769] Sending connection context
0620 23:02:07.853715 (+ 362us) client_negotiation.cc:240] Negotiation successful
0620 23:02:07.854258 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17370,"thread_start_us":20773,"threads_started":1}
I20260620 23:02:07.856918 16345 heartbeater.cc:347] Connected to a master server at 127.2.237.62:46157
I20260620 23:02:07.857244 16345 heartbeater.cc:464] Registering TS with master...
I20260620 23:02:07.857966 16345 heartbeater.cc:511] Master 127.2.237.62:46157 requested a full tablet report, sending...
I20260620 23:02:07.859506 15868 ts_manager.cc:194] Registered new tserver with Master: 6cdcbf7b54604d449e5240f55427ff72 (127.2.237.3:36169)
I20260620 23:02:07.860306 15868 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.3@KRBTEST.COM'} at 127.2.237.3:55083
I20260620 23:02:07.866665 2996 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:02:07.883127 16351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:07.869825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53420 (local address 127.2.237.62:46157)
0620 23:02:07.870005 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:07.870009 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:07.870360 (+ 351us) server_negotiation.cc:407] Connection header received
0620 23:02:07.870509 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:07.870513 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:07.870634 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:07.870727 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:07.872268 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:07.873096 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:07.874486 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:07.874985 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:07.875087 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:07.877676 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:07.877681 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:07.877685 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:07.877719 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:07.880607 (+ 2888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:07.881279 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:07.881283 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:07.881285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:07.881407 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:07.881756 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:07.881759 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:07.881761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:07.882027 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:07.882190 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:07.882554 (+ 364us) server_negotiation.cc:299] Negotiation successful
0620 23:02:07.882824 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260620 23:02:07.888387 15868 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53420:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:02:07.896253 15868 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:02:07.924467 16369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:07.912439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:35857 (local address 127.0.0.1:50414)
0620 23:02:07.913230 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:07.913252 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:07.913373 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:07.914741 (+ 1368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:07.914748 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:07.914788 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:02:07.915313 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:07.915329 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:07.917982 (+ 2653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:07.917990 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:07.921143 (+ 3153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:07.921157 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:07.923147 (+ 1990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:07.923154 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:07.923478 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:07.923539 (+ 61us) client_negotiation.cc:769] Sending connection context
0620 23:02:07.923672 (+ 133us) client_negotiation.cc:240] Negotiation successful
0620 23:02:07.923867 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"mutex_wait_us":94,"thread_start_us":530,"threads_started":1}
I20260620 23:02:07.925578 16370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:07.912680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50414 (local address 127.2.237.1:35857)
0620 23:02:07.913637 (+ 957us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:07.913646 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:07.913673 (+ 27us) server_negotiation.cc:407] Connection header received
0620 23:02:07.913796 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:07.913808 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:07.914165 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:07.914402 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:02:07.915592 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:07.917749 (+ 2157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:07.921347 (+ 3598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:07.922916 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:07.923050 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:07.923288 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:07.925022 (+ 1734us) server_negotiation.cc:299] Negotiation successful
0620 23:02:07.925257 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":800,"thread_start_us":1771,"threads_started":1}
I20260620 23:02:07.926445 16367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:07.911846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:46355 (local address 127.0.0.1:53070)
0620 23:02:07.912826 (+ 980us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:07.912878 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:07.913090 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:07.914502 (+ 1412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:07.914510 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:07.914548 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:02:07.916807 (+ 2259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:07.916828 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:07.921054 (+ 4226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:07.921062 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:07.923395 (+ 2333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:07.923406 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:07.925289 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:07.925296 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:07.925559 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:07.925596 (+ 37us) client_negotiation.cc:769] Sending connection context
0620 23:02:07.925884 (+ 288us) client_negotiation.cc:240] Negotiation successful
0620 23:02:07.926044 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"mutex_wait_us":1664,"thread_start_us":945,"threads_started":1}
I20260620 23:02:07.926458 16368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:07.912005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53070 (local address 127.2.237.2:46355)
0620 23:02:07.913848 (+ 1843us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:07.913856 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:07.913883 (+ 27us) server_negotiation.cc:407] Connection header received
0620 23:02:07.913982 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:07.913991 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:07.914252 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:07.917075 (+ 2823us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:02:07.917417 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:07.920890 (+ 3473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:07.923554 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:07.925023 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:07.925440 (+ 417us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:07.925651 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:07.925980 (+ 329us) server_negotiation.cc:299] Negotiation successful
0620 23:02:07.926149 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1693,"thread_start_us":1558,"threads_started":1}
I20260620 23:02:07.928920 16371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:07.913661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:36169 (local address 127.0.0.1:50054)
0620 23:02:07.920698 (+ 7037us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:07.920717 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:07.920835 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:07.921692 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:07.921697 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:07.921726 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:02:07.922173 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:07.922183 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:07.923975 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:07.923980 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:07.926121 (+ 2141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:07.926138 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:07.928111 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:07.928117 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:07.928293 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:07.928335 (+ 42us) client_negotiation.cc:769] Sending connection context
0620 23:02:07.928422 (+ 87us) client_negotiation.cc:240] Negotiation successful
0620 23:02:07.928557 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6899,"mutex_wait_us":3,"thread_start_us":6700,"threads_started":1}
I20260620 23:02:07.929987 16001 tablet_service.cc:1467] Processing CreateTablet for tablet e372ac668e244199a6792739efbc7cae (DEFAULT_TABLE table=test-table [id=8aac80f2c751443db3b9c2e64227dc0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:02:07.930536 16146 tablet_service.cc:1467] Processing CreateTablet for tablet e372ac668e244199a6792739efbc7cae (DEFAULT_TABLE table=test-table [id=8aac80f2c751443db3b9c2e64227dc0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:02:07.931427 16001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e372ac668e244199a6792739efbc7cae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:02:07.931883 16146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e372ac668e244199a6792739efbc7cae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:02:07.935570 16372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:07.914242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50054 (local address 127.2.237.3:36169)
0620 23:02:07.921017 (+ 6775us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:07.921028 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:07.921067 (+ 39us) server_negotiation.cc:407] Connection header received
0620 23:02:07.921180 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:07.921191 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:07.921473 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:07.921717 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:02:07.922381 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:07.923816 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:07.926312 (+ 2496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:07.927885 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:07.928026 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:07.934884 (+ 6858us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:07.934992 (+ 108us) server_negotiation.cc:299] Negotiation successful
0620 23:02:07.935187 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6571,"thread_start_us":6667,"threads_started":1}
I20260620 23:02:07.940267 16280 tablet_service.cc:1467] Processing CreateTablet for tablet e372ac668e244199a6792739efbc7cae (DEFAULT_TABLE table=test-table [id=8aac80f2c751443db3b9c2e64227dc0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:02:07.942179 16280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e372ac668e244199a6792739efbc7cae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:02:07.950992 16374 tablet_bootstrap.cc:492] T e372ac668e244199a6792739efbc7cae P 2944fc65ae9f438a8ba30b0700840c43: Bootstrap starting.
I20260620 23:02:07.953459 16373 tablet_bootstrap.cc:492] T e372ac668e244199a6792739efbc7cae P 1cd1f46e108c4a0eaf63842d38744fee: Bootstrap starting.
I20260620 23:02:07.956766 16374 tablet_bootstrap.cc:654] T e372ac668e244199a6792739efbc7cae P 2944fc65ae9f438a8ba30b0700840c43: Neither blocks nor log segments found. Creating new log.
I20260620 23:02:07.957517 16373 tablet_bootstrap.cc:654] T e372ac668e244199a6792739efbc7cae P 1cd1f46e108c4a0eaf63842d38744fee: Neither blocks nor log segments found. Creating new log.
I20260620 23:02:07.958240 16374 log.cc:824] T e372ac668e244199a6792739efbc7cae P 2944fc65ae9f438a8ba30b0700840c43: Log is configured to *not* fsync() on all Append() calls
I20260620 23:02:07.958904 16373 log.cc:824] T e372ac668e244199a6792739efbc7cae P 1cd1f46e108c4a0eaf63842d38744fee: Log is configured to *not* fsync() on all Append() calls
I20260620 23:02:07.960563 16377 tablet_bootstrap.cc:492] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72: Bootstrap starting.
I20260620 23:02:07.962250 16374 tablet_bootstrap.cc:492] T e372ac668e244199a6792739efbc7cae P 2944fc65ae9f438a8ba30b0700840c43: No bootstrap required, opened a new log
I20260620 23:02:07.962646 16374 ts_tablet_manager.cc:1397] T e372ac668e244199a6792739efbc7cae P 2944fc65ae9f438a8ba30b0700840c43: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260620 23:02:07.963331 16373 tablet_bootstrap.cc:492] T e372ac668e244199a6792739efbc7cae P 1cd1f46e108c4a0eaf63842d38744fee: No bootstrap required, opened a new log
I20260620 23:02:07.963624 16373 ts_tablet_manager.cc:1397] T e372ac668e244199a6792739efbc7cae P 1cd1f46e108c4a0eaf63842d38744fee: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.003s
I20260620 23:02:07.964959 16377 tablet_bootstrap.cc:654] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72: Neither blocks nor log segments found. Creating new log.
I20260620 23:02:07.966409 16377 log.cc:824] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72: Log is configured to *not* fsync() on all Append() calls
I20260620 23:02:07.972751 16373 raft_consensus.cc:348] T e372ac668e244199a6792739efbc7cae P 1cd1f46e108c4a0eaf63842d38744fee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2944fc65ae9f438a8ba30b0700840c43" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46355 } } peers { permanent_uuid: "1cd1f46e108c4a0eaf63842d38744fee" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 35857 } } peers { permanent_uuid: "6cdcbf7b54604d449e5240f55427ff72" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36169 } }
I20260620 23:02:07.973212 16373 raft_consensus.cc:374] T e372ac668e244199a6792739efbc7cae P 1cd1f46e108c4a0eaf63842d38744fee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:02:07.973325 16373 raft_consensus.cc:729] T e372ac668e244199a6792739efbc7cae P 1cd1f46e108c4a0eaf63842d38744fee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cd1f46e108c4a0eaf63842d38744fee, State: Initialized, Role: FOLLOWER
I20260620 23:02:07.974180 16377 tablet_bootstrap.cc:492] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72: No bootstrap required, opened a new log
I20260620 23:02:07.974061 16373 consensus_queue.cc:260] T e372ac668e244199a6792739efbc7cae P 1cd1f46e108c4a0eaf63842d38744fee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2944fc65ae9f438a8ba30b0700840c43" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46355 } } peers { permanent_uuid: "1cd1f46e108c4a0eaf63842d38744fee" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 35857 } } peers { permanent_uuid: "6cdcbf7b54604d449e5240f55427ff72" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36169 } }
I20260620 23:02:07.974550 16377 ts_tablet_manager.cc:1397] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.007s
I20260620 23:02:07.975409 16066 heartbeater.cc:503] Master 127.2.237.62:46157 was elected leader, sending a full tablet report...
I20260620 23:02:07.975466 16374 raft_consensus.cc:348] T e372ac668e244199a6792739efbc7cae P 2944fc65ae9f438a8ba30b0700840c43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2944fc65ae9f438a8ba30b0700840c43" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46355 } } peers { permanent_uuid: "1cd1f46e108c4a0eaf63842d38744fee" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 35857 } } peers { permanent_uuid: "6cdcbf7b54604d449e5240f55427ff72" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36169 } }
I20260620 23:02:07.975952 16374 raft_consensus.cc:374] T e372ac668e244199a6792739efbc7cae P 2944fc65ae9f438a8ba30b0700840c43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:02:07.976058 16374 raft_consensus.cc:729] T e372ac668e244199a6792739efbc7cae P 2944fc65ae9f438a8ba30b0700840c43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2944fc65ae9f438a8ba30b0700840c43, State: Initialized, Role: FOLLOWER
I20260620 23:02:07.976176 16373 ts_tablet_manager.cc:1428] T e372ac668e244199a6792739efbc7cae P 1cd1f46e108c4a0eaf63842d38744fee: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260620 23:02:07.976723 16374 consensus_queue.cc:260] T e372ac668e244199a6792739efbc7cae P 2944fc65ae9f438a8ba30b0700840c43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2944fc65ae9f438a8ba30b0700840c43" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46355 } } peers { permanent_uuid: "1cd1f46e108c4a0eaf63842d38744fee" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 35857 } } peers { permanent_uuid: "6cdcbf7b54604d449e5240f55427ff72" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36169 } }
I20260620 23:02:07.978848 16211 heartbeater.cc:503] Master 127.2.237.62:46157 was elected leader, sending a full tablet report...
I20260620 23:02:07.979729 16374 ts_tablet_manager.cc:1428] T e372ac668e244199a6792739efbc7cae P 2944fc65ae9f438a8ba30b0700840c43: Time spent starting tablet: real 0.017s user 0.010s sys 0.003s
I20260620 23:02:07.987126 16377 raft_consensus.cc:348] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2944fc65ae9f438a8ba30b0700840c43" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46355 } } peers { permanent_uuid: "1cd1f46e108c4a0eaf63842d38744fee" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 35857 } } peers { permanent_uuid: "6cdcbf7b54604d449e5240f55427ff72" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36169 } }
I20260620 23:02:07.987596 16377 raft_consensus.cc:374] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:02:07.987720 16377 raft_consensus.cc:729] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cdcbf7b54604d449e5240f55427ff72, State: Initialized, Role: FOLLOWER
I20260620 23:02:07.988384 16377 consensus_queue.cc:260] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2944fc65ae9f438a8ba30b0700840c43" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46355 } } peers { permanent_uuid: "1cd1f46e108c4a0eaf63842d38744fee" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 35857 } } peers { permanent_uuid: "6cdcbf7b54604d449e5240f55427ff72" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36169 } }
I20260620 23:02:07.989454 16345 heartbeater.cc:503] Master 127.2.237.62:46157 was elected leader, sending a full tablet report...
I20260620 23:02:07.989971 16377 ts_tablet_manager.cc:1428] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72: Time spent starting tablet: real 0.015s user 0.005s sys 0.007s
I20260620 23:02:08.023706 16381 raft_consensus.cc:482] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:02:08.023974 16381 raft_consensus.cc:504] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2944fc65ae9f438a8ba30b0700840c43" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46355 } } peers { permanent_uuid: "1cd1f46e108c4a0eaf63842d38744fee" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 35857 } } peers { permanent_uuid: "6cdcbf7b54604d449e5240f55427ff72" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36169 } }
I20260620 23:02:08.025606 16381 leader_election.cc:288] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2944fc65ae9f438a8ba30b0700840c43 (127.2.237.2:46355), 1cd1f46e108c4a0eaf63842d38744fee (127.2.237.1:35857)
I20260620 23:02:08.031701 16368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:08.026209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:33917 (local address 127.2.237.2:46355)
0620 23:02:08.026439 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:08.026446 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:08.026605 (+ 159us) server_negotiation.cc:407] Connection header received
0620 23:02:08.026858 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:08.026863 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:08.026947 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:08.027052 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:02:08.027495 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:08.028187 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:08.029839 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:08.030784 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:08.030942 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:08.031036 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:08.031310 (+ 274us) server_negotiation.cc:299] Negotiation successful
0620 23:02:08.031445 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260620 23:02:08.031741 16349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:08.026156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:46355 (local address 127.2.237.3:33917)
0620 23:02:08.026483 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:08.026500 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:08.026696 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:08.027074 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:08.027078 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:08.027104 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:02:08.027324 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:08.027332 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:08.028332 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:08.028338 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:08.029653 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:08.029664 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:08.030957 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:08.030963 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:08.031144 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:08.031168 (+ 24us) client_negotiation.cc:769] Sending connection context
0620 23:02:08.031287 (+ 119us) client_negotiation.cc:240] Negotiation successful
0620 23:02:08.031426 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":4}
I20260620 23:02:08.032310 16370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:08.026128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:56875 (local address 127.2.237.1:35857)
0620 23:02:08.026392 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:08.026398 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:08.027168 (+ 770us) server_negotiation.cc:407] Connection header received
0620 23:02:08.027382 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:08.027386 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:08.027464 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:08.027566 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:02:08.028099 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:08.028800 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:08.030467 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:08.031355 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:08.031509 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:08.031606 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:08.031942 (+ 336us) server_negotiation.cc:299] Negotiation successful
0620 23:02:08.032068 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260620 23:02:08.033042 16382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:08.026214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:35857 (local address 127.2.237.3:56875)
0620 23:02:08.027038 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:08.027052 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:08.027218 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:08.027589 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:08.027593 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:08.027617 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:02:08.027927 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:08.027934 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:08.028963 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:08.028968 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:08.030291 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:08.030302 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:08.031568 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:08.031574 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:08.031730 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:08.031752 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 23:02:08.032561 (+ 809us) client_negotiation.cc:240] Negotiation successful
0620 23:02:08.032705 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"thread_start_us":498,"threads_started":1}
I20260620 23:02:08.032912 16166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e372ac668e244199a6792739efbc7cae" candidate_uuid: "6cdcbf7b54604d449e5240f55427ff72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2944fc65ae9f438a8ba30b0700840c43" is_pre_election: true
I20260620 23:02:08.033568 16166 raft_consensus.cc:2454] T e372ac668e244199a6792739efbc7cae P 2944fc65ae9f438a8ba30b0700840c43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cdcbf7b54604d449e5240f55427ff72 in term 0.
I20260620 23:02:08.034106 16021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e372ac668e244199a6792739efbc7cae" candidate_uuid: "6cdcbf7b54604d449e5240f55427ff72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cd1f46e108c4a0eaf63842d38744fee" is_pre_election: true
I20260620 23:02:08.034360 16235 leader_election.cc:302] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2944fc65ae9f438a8ba30b0700840c43, 6cdcbf7b54604d449e5240f55427ff72; no voters:
I20260620 23:02:08.034685 16021 raft_consensus.cc:2454] T e372ac668e244199a6792739efbc7cae P 1cd1f46e108c4a0eaf63842d38744fee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cdcbf7b54604d449e5240f55427ff72 in term 0.
I20260620 23:02:08.034864 16381 raft_consensus.cc:2781] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:02:08.035023 16381 raft_consensus.cc:482] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:02:08.035135 16381 raft_consensus.cc:3037] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:02:08.037333 16381 raft_consensus.cc:504] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2944fc65ae9f438a8ba30b0700840c43" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46355 } } peers { permanent_uuid: "1cd1f46e108c4a0eaf63842d38744fee" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 35857 } } peers { permanent_uuid: "6cdcbf7b54604d449e5240f55427ff72" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36169 } }
I20260620 23:02:08.038074 16381 leader_election.cc:288] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72 [CANDIDATE]: Term 1 election: Requested vote from peers 2944fc65ae9f438a8ba30b0700840c43 (127.2.237.2:46355), 1cd1f46e108c4a0eaf63842d38744fee (127.2.237.1:35857)
I20260620 23:02:08.038511 16166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e372ac668e244199a6792739efbc7cae" candidate_uuid: "6cdcbf7b54604d449e5240f55427ff72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2944fc65ae9f438a8ba30b0700840c43"
I20260620 23:02:08.038683 16021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e372ac668e244199a6792739efbc7cae" candidate_uuid: "6cdcbf7b54604d449e5240f55427ff72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cd1f46e108c4a0eaf63842d38744fee"
I20260620 23:02:08.038789 16166 raft_consensus.cc:3037] T e372ac668e244199a6792739efbc7cae P 2944fc65ae9f438a8ba30b0700840c43 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:02:08.038960 16021 raft_consensus.cc:3037] T e372ac668e244199a6792739efbc7cae P 1cd1f46e108c4a0eaf63842d38744fee [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:02:08.041654 16021 raft_consensus.cc:2454] T e372ac668e244199a6792739efbc7cae P 1cd1f46e108c4a0eaf63842d38744fee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cdcbf7b54604d449e5240f55427ff72 in term 1.
I20260620 23:02:08.041860 16166 raft_consensus.cc:2454] T e372ac668e244199a6792739efbc7cae P 2944fc65ae9f438a8ba30b0700840c43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cdcbf7b54604d449e5240f55427ff72 in term 1.
I20260620 23:02:08.042178 16233 leader_election.cc:302] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cd1f46e108c4a0eaf63842d38744fee, 6cdcbf7b54604d449e5240f55427ff72; no voters:
I20260620 23:02:08.042564 16381 raft_consensus.cc:2781] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:02:08.042997 16381 raft_consensus.cc:686] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72 [term 1 LEADER]: Becoming Leader. State: Replica: 6cdcbf7b54604d449e5240f55427ff72, State: Running, Role: LEADER
I20260620 23:02:08.043588 16381 consensus_queue.cc:237] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2944fc65ae9f438a8ba30b0700840c43" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46355 } } peers { permanent_uuid: "1cd1f46e108c4a0eaf63842d38744fee" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 35857 } } peers { permanent_uuid: "6cdcbf7b54604d449e5240f55427ff72" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36169 } }
I20260620 23:02:08.048628 15867 catalog_manager.cc:5495] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cdcbf7b54604d449e5240f55427ff72 (127.2.237.3). New cstate: current_term: 1 leader_uuid: "6cdcbf7b54604d449e5240f55427ff72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2944fc65ae9f438a8ba30b0700840c43" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cd1f46e108c4a0eaf63842d38744fee" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 35857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6cdcbf7b54604d449e5240f55427ff72" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 36169 } health_report { overall_health: HEALTHY } } }
W20260620 23:02:08.082231 16348 tablet.cc:2369] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:02:08.085148 16372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:08.079501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50060 (local address 127.2.237.3:36169)
0620 23:02:08.079725 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:08.079730 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:08.079753 (+ 23us) server_negotiation.cc:407] Connection header received
0620 23:02:08.079993 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:08.079997 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:08.080078 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:08.080217 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:02:08.080598 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:08.081376 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:08.082899 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:08.083317 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:08.083463 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:08.084002 (+ 539us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:02:08.084539 (+ 537us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:02:08.084706 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:08.084785 (+ 79us) server_negotiation.cc:299] Negotiation successful
0620 23:02:08.084872 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260620 23:02:08.094223 16166 raft_consensus.cc:1261] T e372ac668e244199a6792739efbc7cae P 2944fc65ae9f438a8ba30b0700840c43 [term 1 FOLLOWER]: Refusing update from remote peer 6cdcbf7b54604d449e5240f55427ff72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:02:08.094215 16021 raft_consensus.cc:1261] T e372ac668e244199a6792739efbc7cae P 1cd1f46e108c4a0eaf63842d38744fee [term 1 FOLLOWER]: Refusing update from remote peer 6cdcbf7b54604d449e5240f55427ff72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:02:08.095418 16383 consensus_queue.cc:1035] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cd1f46e108c4a0eaf63842d38744fee" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 35857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:02:08.095829 16381 consensus_queue.cc:1035] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2944fc65ae9f438a8ba30b0700840c43" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 23:02:08.108027 16067 tablet.cc:2369] T e372ac668e244199a6792739efbc7cae P 1cd1f46e108c4a0eaf63842d38744fee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:02:08.112668 16388 mvcc.cc:205] Tried to move back new op lower bound from 7299057779063148544 to 7299057778871070720. Current Snapshot: MvccSnapshot[applied={T|T < 7299057779063148544}]
I20260620 23:02:08.135972 15866 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:02:08.136336 15866 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:02:08.140759 15866 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e6b9b60d17674c028cf50bde5e8e9dba: Sending DeleteTablet for 3 replicas of tablet e372ac668e244199a6792739efbc7cae
I20260620 23:02:08.142786 16001 tablet_service.cc:1514] Processing DeleteTablet for tablet e372ac668e244199a6792739efbc7cae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:02:08 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:50414
I20260620 23:02:08.143184 16146 tablet_service.cc:1514] Processing DeleteTablet for tablet e372ac668e244199a6792739efbc7cae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:02:08 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:53070
I20260620 23:02:08.144035 16396 tablet_replica.cc:330] T e372ac668e244199a6792739efbc7cae P 1cd1f46e108c4a0eaf63842d38744fee: stopping tablet replica
I20260620 23:02:08.143973 16280 tablet_service.cc:1514] Processing DeleteTablet for tablet e372ac668e244199a6792739efbc7cae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:02:08 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:50054
I20260620 23:02:08.144590 16396 raft_consensus.cc:2229] T e372ac668e244199a6792739efbc7cae P 1cd1f46e108c4a0eaf63842d38744fee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:02:08.145169 16396 raft_consensus.cc:2258] T e372ac668e244199a6792739efbc7cae P 1cd1f46e108c4a0eaf63842d38744fee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:02:08.147444 16396 ts_tablet_manager.cc:1905] T e372ac668e244199a6792739efbc7cae P 1cd1f46e108c4a0eaf63842d38744fee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:02:08.149684 16397 tablet_replica.cc:330] T e372ac668e244199a6792739efbc7cae P 2944fc65ae9f438a8ba30b0700840c43: stopping tablet replica
I20260620 23:02:08.150179 16397 raft_consensus.cc:2229] T e372ac668e244199a6792739efbc7cae P 2944fc65ae9f438a8ba30b0700840c43 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:02:08.150812 16397 raft_consensus.cc:2258] T e372ac668e244199a6792739efbc7cae P 2944fc65ae9f438a8ba30b0700840c43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:02:08.151263 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 15905
I20260620 23:02:08.156615 16396 ts_tablet_manager.cc:1918] T e372ac668e244199a6792739efbc7cae P 1cd1f46e108c4a0eaf63842d38744fee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:02:08.157513 16396 log.cc:1196] T e372ac668e244199a6792739efbc7cae P 1cd1f46e108c4a0eaf63842d38744fee: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-0/wal/wals/e372ac668e244199a6792739efbc7cae
I20260620 23:02:08.158226 16396 ts_tablet_manager.cc:1939] T e372ac668e244199a6792739efbc7cae P 1cd1f46e108c4a0eaf63842d38744fee: Deleting consensus metadata
I20260620 23:02:08.158651 16397 ts_tablet_manager.cc:1905] T e372ac668e244199a6792739efbc7cae P 2944fc65ae9f438a8ba30b0700840c43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:02:08.160763 15853 catalog_manager.cc:4841] TS 1cd1f46e108c4a0eaf63842d38744fee (127.2.237.1:35857): tablet e372ac668e244199a6792739efbc7cae (table test-table [id=8aac80f2c751443db3b9c2e64227dc0b]) successfully deleted
I20260620 23:02:08.166041 16397 ts_tablet_manager.cc:1918] T e372ac668e244199a6792739efbc7cae P 2944fc65ae9f438a8ba30b0700840c43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:02:08.166230 16397 log.cc:1196] T e372ac668e244199a6792739efbc7cae P 2944fc65ae9f438a8ba30b0700840c43: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-1/wal/wals/e372ac668e244199a6792739efbc7cae
I20260620 23:02:08.166313 16398 tablet_replica.cc:330] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72: stopping tablet replica
I20260620 23:02:08.166832 16397 ts_tablet_manager.cc:1939] T e372ac668e244199a6792739efbc7cae P 2944fc65ae9f438a8ba30b0700840c43: Deleting consensus metadata
I20260620 23:02:08.166869 16398 raft_consensus.cc:2229] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72 [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:02:08.167611 16398 raft_consensus.cc:2258] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:02:08.169036 15855 catalog_manager.cc:4841] TS 2944fc65ae9f438a8ba30b0700840c43 (127.2.237.2:46355): tablet e372ac668e244199a6792739efbc7cae (table test-table [id=8aac80f2c751443db3b9c2e64227dc0b]) successfully deleted
I20260620 23:02:08.170717 16398 ts_tablet_manager.cc:1905] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:02:08.173026 16370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:08.153041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50426 (local address 127.2.237.1:35857)
0620 23:02:08.153224 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:08.153230 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:08.153249 (+ 19us) server_negotiation.cc:407] Connection header received
0620 23:02:08.153314 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:08.153320 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:08.153411 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:08.153524 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:08.156157 (+ 2633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:08.157386 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:08.159457 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:08.160124 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:08.160259 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:08.164507 (+ 4248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:08.164524 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:08.164545 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:08.164599 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:08.169491 (+ 4892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:08.170561 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:08.170649 (+ 88us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:08.170652 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:08.170885 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:08.171326 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:08.171331 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:08.171334 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:08.171713 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:08.172042 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:08.172407 (+ 365us) server_negotiation.cc:299] Negotiation successful
0620 23:02:08.172636 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260620 23:02:08.173821 16061 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:02:08.178548 16398 ts_tablet_manager.cc:1918] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:02:08.178788 16398 log.cc:1196] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781996318807807-2996-0/minicluster-data/ts-2/wal/wals/e372ac668e244199a6792739efbc7cae
I20260620 23:02:08.179384 16398 ts_tablet_manager.cc:1939] T e372ac668e244199a6792739efbc7cae P 6cdcbf7b54604d449e5240f55427ff72: Deleting consensus metadata
I20260620 23:02:08.181447 15855 catalog_manager.cc:4841] TS 6cdcbf7b54604d449e5240f55427ff72 (127.2.237.3:36169): tablet e372ac668e244199a6792739efbc7cae (table test-table [id=8aac80f2c751443db3b9c2e64227dc0b]) successfully deleted
I20260620 23:02:08.268620 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 15905
I20260620 23:02:08.287129 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 16070
I20260620 23:02:08.303134 16368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:08.288109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53086 (local address 127.2.237.2:46355)
0620 23:02:08.288302 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:08.288311 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:08.288382 (+ 71us) server_negotiation.cc:407] Connection header received
0620 23:02:08.288640 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:08.288645 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:08.288719 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:08.288845 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:08.290333 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:08.291213 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:08.292621 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:08.293052 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:08.293220 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:08.296139 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:08.296158 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:08.296175 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:08.296221 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:08.299694 (+ 3473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:08.300566 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:08.300581 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:08.300583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:08.300746 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:08.301230 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:08.301236 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:08.301240 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:08.301680 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:08.302014 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:08.302399 (+ 385us) server_negotiation.cc:299] Negotiation successful
0620 23:02:08.302690 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260620 23:02:08.303972 16206 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:02:08.398137 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 16070
I20260620 23:02:08.416507 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 16215
I20260620 23:02:08.433832 16372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:08.417484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50072 (local address 127.2.237.3:36169)
0620 23:02:08.417695 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:08.417701 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:08.417726 (+ 25us) server_negotiation.cc:407] Connection header received
0620 23:02:08.417898 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:08.417910 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:08.418009 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:08.418133 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:08.419662 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:08.420981 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:08.422509 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:08.423204 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:08.423357 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:08.426209 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:08.426224 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:08.426246 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:08.426323 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:08.430600 (+ 4277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:08.431499 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:08.431510 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:08.431519 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:08.431729 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:08.432118 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:08.432125 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:08.432134 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:08.432619 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:08.432978 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:08.433144 (+ 166us) server_negotiation.cc:299] Negotiation successful
0620 23:02:08.433398 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260620 23:02:08.434719 16340 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:02:08.557705 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 16215
I20260620 23:02:08.577395 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 15835
I20260620 23:02:08.578727 15898 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:02:08.693818 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 15835
I20260620 23:02:08.721540 2996 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260620 23:02:09.057286 2996 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260620 23:02:09.057551 2996 mini_ranger.cc:66] Stopping Ranger...
I20260620 23:02:09.404031 2996 mini_ranger.cc:68] Stopped Ranger
2026-06-20 23:02:09.404 UTC [14887] LOG: received smart shutdown request
2026-06-20 23:02:09.407 UTC [14887] LOG: background worker "logical replication launcher" (PID 14896) exited with exit code 1
2026-06-20 23:02:09.407 UTC [14891] LOG: shutting down
2026-06-20 23:02:09.590 UTC [14887] LOG: database system is shut down
2026-06-20T23:02:09Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (56254 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:02:09 dist-test-slave-f63b krb5kdc[16411](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:02:09 dist-test-slave-f63b krb5kdc[16411](info): set up 2 sockets
Jun 20 23:02:09 dist-test-slave-f63b krb5kdc[16411](info): commencing operation
krb5kdc: starting...
W20260620 23:02:10.847070 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 23:02:10 dist-test-slave-f63b krb5kdc[16411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:02:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:02:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
I20260620 23:02:11.090667 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:44639
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:41853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:44639,127.2.237.61:39863,127.2.237.60:45945
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:02:11.437754 16424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:02:11.438102 16424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:02:11.438282 16424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:02:11.446089 16424 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:02:11.446218 16424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:02:11.446321 16424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:02:11.446441 16424 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:02:11.446535 16424 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:02:11.457441 16424 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:41853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:44639,127.2.237.61:39863,127.2.237.60:45945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:44639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:02:11.459066 16424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:02:11.460495 16424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:02:11.473460 16432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:11.473460 16430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:11.473737 16429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:02:11.474687 16424 server_base.cc:1034] running on GCE node
I20260620 23:02:11.475593 16424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:02:11.477103 16424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:02:11.478358 16424 hybrid_clock.cc:648] HybridClock initialized: now 1781996531478319 us; error 60 us; skew 500 ppm
I20260620 23:02:11.478817 16424 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:02:11 dist-test-slave-f63b krb5kdc[16411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996531, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:02:11.485741 16424 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 23:02:11.487669 16424 webserver.cc:460] Webserver started at http://127.2.237.62:36735/ using document root <none> and password file <none>
I20260620 23:02:11.488478 16424 fs_manager.cc:359] Metadata directory not provided
I20260620 23:02:11.488648 16424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:02:11.489002 16424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:02:11.492379 16424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "bfb1d92e25794bbaaec541472c7682a1"
format_stamp: "Formatted at 2026-06-20 23:02:11 on dist-test-slave-f63b"
server_key: "0753f6bfe4aebac0f010bfbbe41e2ca9"
server_key_iv: "8a32d3241238b982dbe270e4caeb61bf"
server_key_version: "encryptionkey@0"
I20260620 23:02:11.493345 16424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "bfb1d92e25794bbaaec541472c7682a1"
format_stamp: "Formatted at 2026-06-20 23:02:11 on dist-test-slave-f63b"
server_key: "0753f6bfe4aebac0f010bfbbe41e2ca9"
server_key_iv: "8a32d3241238b982dbe270e4caeb61bf"
server_key_version: "encryptionkey@0"
I20260620 23:02:11.499310 16424 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260620 23:02:11.503145 16439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:11.504671 16424 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260620 23:02:11.504904 16424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "bfb1d92e25794bbaaec541472c7682a1"
format_stamp: "Formatted at 2026-06-20 23:02:11 on dist-test-slave-f63b"
server_key: "0753f6bfe4aebac0f010bfbbe41e2ca9"
server_key_iv: "8a32d3241238b982dbe270e4caeb61bf"
server_key_version: "encryptionkey@0"
I20260620 23:02:11.505198 16424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:02:11.531867 16424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:02:11.536489 16424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:02:11.536818 16424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:02:11.556648 16424 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:44639
I20260620 23:02:11.556670 16490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:44639 every 8 connection(s)
I20260620 23:02:11.558167 16424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:02:11.562088 16491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:02:11.566598 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 16424
I20260620 23:02:11.566840 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:02:11.567654 2996 external_mini_cluster.cc:1428] Setting key 2d79dc95ce8490eada3a9591ce340683
I20260620 23:02:11.570966 16491 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44639 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:02:11 dist-test-slave-f63b krb5kdc[16411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996531, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.237.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.61@KRBTEST.COM" created.
I20260620 23:02:11.602128 16493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:11.575358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:44639 (local address 127.0.0.1:60714)
0620 23:02:11.576347 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:11.576398 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:11.579014 (+ 2616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:11.580005 (+ 991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:11.580019 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:11.580876 (+ 857us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:02:11.581422 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:11.581451 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:11.583140 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:11.583148 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:11.585101 (+ 1953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:11.585121 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:11.586142 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:11.586156 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:11.586392 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:11.587163 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:02:11.587200 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:02:11.590113 (+ 2913us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:02:11.593876 (+ 3763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:11.593891 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:11.593906 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:11.594460 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:11.594994 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:11.595000 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:11.595004 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:11.595204 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:11.599644 (+ 4440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:02:11.599656 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:02:11.599972 (+ 316us) client_negotiation.cc:769] Sending connection context
0620 23:02:11.600303 (+ 331us) client_negotiation.cc:240] Negotiation successful
0620 23:02:11.601071 (+ 768us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685,"mutex_wait_us":162,"thread_start_us":424,"threads_started":1}
I20260620 23:02:11.602159 16494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:11.575359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60714 (local address 127.2.237.62:44639)
0620 23:02:11.576566 (+ 1207us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:11.576576 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:11.576634 (+ 58us) server_negotiation.cc:407] Connection header received
0620 23:02:11.579441 (+ 2807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:11.579447 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:11.579714 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:11.579890 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:11.581650 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:11.582969 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:11.585275 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:11.585865 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:11.586005 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:11.590385 (+ 4380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:11.590394 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:11.590409 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:11.590441 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:11.593701 (+ 3260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:11.594643 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:11.594655 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:11.594664 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:11.594866 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:11.598736 (+ 3870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:11.598741 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:11.598744 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:11.599418 (+ 674us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:11.599614 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:11.600469 (+ 855us) server_negotiation.cc:299] Negotiation successful
0620 23:02:11.600847 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":983,"spinlock_wait_cycles":19328,"thread_start_us":771,"threads_started":1}
I20260620 23:02:11.605345 16491 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.61" port: 39863 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:02:11.606508 16493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:11.606093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:39863 (local address 127.0.0.1:59522)
0620 23:02:11.606278 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:11.606393 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.61:39863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260620 23:02:11.606905 16442 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.237.61:39863: connect: Connection refused (error 111)
I20260620 23:02:11.608623 16493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:11.608237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:39863 (local address 127.0.0.1:59536)
0620 23:02:11.608410 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:11.608489 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.61:39863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260620 23:02:11.608830 16491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.61:39863: Network error: Client connection negotiation failed: client connection to 127.2.237.61:39863: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.61.keytab.
Entry for principal kudu/127.2.237.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.61.keytab.
I20260620 23:02:11.626986 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.61:39863
--webserver_interface=127.2.237.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.61.keytab
--principal=kudu/127.2.237.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:41853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:44639,127.2.237.61:39863,127.2.237.60:45945
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:02:11.658216 16491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.61" port: 39863 } attempt: 1
I20260620 23:02:11.659597 16493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:11.659028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:39863 (local address 127.0.0.1:59552)
0620 23:02:11.659270 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:11.659402 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.61:39863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260620 23:02:11.660825 16493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:11.660350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:39863 (local address 127.0.0.1:59560)
0620 23:02:11.660524 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:11.660615 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.61:39863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260620 23:02:11.661072 16491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.61:39863: Network error: Client connection negotiation failed: client connection to 127.2.237.61:39863: connect: Connection refused (error 111)
I20260620 23:02:11.729403 16491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.61" port: 39863 } attempt: 2
I20260620 23:02:11.730820 16493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:11.730197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:39863 (local address 127.0.0.1:59572)
0620 23:02:11.730472 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:11.730634 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.61:39863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260620 23:02:11.732210 16493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:11.731631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:39863 (local address 127.0.0.1:59580)
0620 23:02:11.731839 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:11.731976 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.61:39863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260620 23:02:11.732506 16491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.61:39863: Network error: Client connection negotiation failed: client connection to 127.2.237.61:39863: connect: Connection refused (error 111)
I20260620 23:02:11.823869 16491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.61" port: 39863 } attempt: 3
I20260620 23:02:11.825354 16493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:11.824779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:39863 (local address 127.0.0.1:59590)
0620 23:02:11.825013 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:11.825152 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.61:39863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260620 23:02:11.826535 16493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:11.826142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:39863 (local address 127.0.0.1:59598)
0620 23:02:11.826290 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:11.826368 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.61:39863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260620 23:02:11.826886 16491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.61:39863: Network error: Client connection negotiation failed: client connection to 127.2.237.61:39863: connect: Connection refused (error 111)
I20260620 23:02:11.970185 16491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.61" port: 39863 } attempt: 4
I20260620 23:02:11.971508 16493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:11.970905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:39863 (local address 127.0.0.1:59612)
0620 23:02:11.971217 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:11.971310 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.61:39863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
I20260620 23:02:11.972623 16493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:11.972243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:39863 (local address 127.0.0.1:59626)
0620 23:02:11.972399 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:11.972500 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.61:39863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260620 23:02:11.972867 16491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.61:39863: Network error: Client connection negotiation failed: client connection to 127.2.237.61:39863: connect: Connection refused (error 111)
W20260620 23:02:11.985839 16498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:02:11.986176 16498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:02:11.986358 16498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:02:11.994230 16498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:02:11.994357 16498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:02:11.994447 16498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:02:11.994560 16498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:02:11.994692 16498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:02:12.005223 16498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:41853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:44639,127.2.237.61:39863,127.2.237.60:45945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.61.keytab
--principal=kudu/127.2.237.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.61:39863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.237.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:02:12.006784 16498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:02:12.008188 16498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:02:12.021239 16504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:12.021298 16506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:12.021217 16503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:02:12.022354 16498 server_base.cc:1034] running on GCE node
I20260620 23:02:12.023298 16498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:02:12.024894 16498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:02:12.026156 16498 hybrid_clock.cc:648] HybridClock initialized: now 1781996532026114 us; error 41 us; skew 500 ppm
I20260620 23:02:12.026707 16498 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:02:12 dist-test-slave-f63b krb5kdc[16411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996532, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:02:12.030373 16498 init.cc:377] Logged in from keytab as kudu/127.2.237.61@KRBTEST.COM (short username kudu)
I20260620 23:02:12.032051 16498 webserver.cc:460] Webserver started at http://127.2.237.61:37973/ using document root <none> and password file <none>
I20260620 23:02:12.032788 16498 fs_manager.cc:359] Metadata directory not provided
I20260620 23:02:12.032895 16498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:02:12.033186 16498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:02:12.036468 16498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-1/data/instance:
uuid: "5e34876e039e4944ba27d633b7a071ce"
format_stamp: "Formatted at 2026-06-20 23:02:12 on dist-test-slave-f63b"
server_key: "19012700a62e6a657f39dacb30bd10f6"
server_key_iv: "e24ca0f805f6a92acec256d124856ac7"
server_key_version: "encryptionkey@0"
I20260620 23:02:12.037322 16498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-1/wal/instance:
uuid: "5e34876e039e4944ba27d633b7a071ce"
format_stamp: "Formatted at 2026-06-20 23:02:12 on dist-test-slave-f63b"
server_key: "19012700a62e6a657f39dacb30bd10f6"
server_key_iv: "e24ca0f805f6a92acec256d124856ac7"
server_key_version: "encryptionkey@0"
I20260620 23:02:12.043051 16498 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260620 23:02:12.046954 16513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:12.048635 16498 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 23:02:12.048851 16498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-1/wal
uuid: "5e34876e039e4944ba27d633b7a071ce"
format_stamp: "Formatted at 2026-06-20 23:02:12 on dist-test-slave-f63b"
server_key: "19012700a62e6a657f39dacb30bd10f6"
server_key_iv: "e24ca0f805f6a92acec256d124856ac7"
server_key_version: "encryptionkey@0"
I20260620 23:02:12.049050 16498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:02:12.065336 16498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:02:12.069878 16498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:02:12.070212 16498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:02:12.089689 16498 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.61:39863
I20260620 23:02:12.089720 16564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.61:39863 every 8 connection(s)
I20260620 23:02:12.091370 16498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-1/data/info.pb
I20260620 23:02:12.095541 16565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:02:12.100972 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 16498
I20260620 23:02:12.101200 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-1/wal/instance
I20260620 23:02:12.101677 2996 external_mini_cluster.cc:1428] Setting key 332b0d2a8c04404f5513f0e11a973adc
I20260620 23:02:12.102862 16565 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44639 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:02:12 dist-test-slave-f63b krb5kdc[16411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996532, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.61@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.237.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.60@KRBTEST.COM" created.
I20260620 23:02:12.127565 16567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.105933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60718 (local address 127.2.237.62:44639)
0620 23:02:12.106456 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:12.106463 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:12.107713 (+ 1250us) server_negotiation.cc:407] Connection header received
0620 23:02:12.110088 (+ 2375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:12.110097 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:12.110226 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:12.110369 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:12.112133 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.113371 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.115201 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.115949 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.116106 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:12.120416 (+ 4310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:12.120421 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:12.120425 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:12.120475 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:12.123503 (+ 3028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:12.124439 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:12.124445 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:12.124447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:12.124601 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:12.125094 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:12.125098 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:12.125100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:12.125395 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:12.125530 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:12.126680 (+ 1150us) server_negotiation.cc:299] Negotiation successful
0620 23:02:12.126977 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":262,"threads_started":1}
I20260620 23:02:12.128137 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.106545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:44639 (local address 127.0.0.1:60718)
0620 23:02:12.107476 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:12.107531 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:12.109723 (+ 2192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:12.110676 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:12.110685 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:12.111268 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:02:12.111919 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:12.111938 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.113563 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.113567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:12.115016 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:12.115031 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.116184 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.116189 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:12.116382 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:12.117168 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:02:12.117193 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:02:12.120086 (+ 2893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:02:12.123679 (+ 3593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:12.123688 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:12.123704 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:12.124286 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:12.124707 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:12.124710 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:12.124712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:12.124961 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:12.125558 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:02:12.125567 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:02:12.126133 (+ 566us) client_negotiation.cc:769] Sending connection context
0620 23:02:12.126398 (+ 265us) client_negotiation.cc:240] Negotiation successful
0620 23:02:12.126901 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":318,"threads_started":1}
I20260620 23:02:12.130112 16565 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.61" port: 39863 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:02:12 dist-test-slave-f63b krb5kdc[16411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996532, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.61@KRBTEST.COM for kudu/127.2.237.61@KRBTEST.COM
I20260620 23:02:12.146377 16570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.131193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59640 (local address 127.2.237.61:39863)
0620 23:02:12.131811 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:12.131818 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:12.131843 (+ 25us) server_negotiation.cc:407] Connection header received
0620 23:02:12.131887 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:12.131893 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:12.132127 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:12.132293 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:12.133052 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.134124 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.135359 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.135901 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.136038 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:12.139793 (+ 3755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:12.139807 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:12.139822 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:12.139852 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:12.142952 (+ 3100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:12.143935 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:12.143942 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:12.143944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:12.144090 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:12.144585 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:12.144589 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:12.144590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:12.144955 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:12.145135 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:12.145651 (+ 516us) server_negotiation.cc:299] Negotiation successful
0620 23:02:12.146015 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":138,"server-negotiator.queue_time_us":463,"thread_start_us":315,"threads_started":1}
I20260620 23:02:12.146473 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.131081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:39863 (local address 127.0.0.1:59640)
0620 23:02:12.131274 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:12.131295 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:12.131433 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:12.132276 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:12.132280 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:12.132756 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:02:12.132896 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:12.132904 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.134280 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.134287 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:12.135213 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:12.135225 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.136047 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.136052 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:12.136276 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:12.136880 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:02:12.136898 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:02:12.139553 (+ 2655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:02:12.143120 (+ 3567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:12.143128 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:12.143135 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:12.143779 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:12.144237 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:12.144245 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:12.144251 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:12.144467 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:12.145101 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:02:12.145109 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:02:12.145321 (+ 212us) client_negotiation.cc:769] Sending connection context
0620 23:02:12.145522 (+ 201us) client_negotiation.cc:240] Negotiation successful
0620 23:02:12.145887 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260620 23:02:12.149089 16565 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:02:12.150449 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.149943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:45945 (local address 127.0.0.1:49598)
0620 23:02:12.150128 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:12.150248 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260620 23:02:12.150784 16515 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.237.60:45945: connect: Connection refused (error 111)
I20260620 23:02:12.152521 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.152117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:45945 (local address 127.0.0.1:49604)
0620 23:02:12.152268 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:12.152397 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260620 23:02:12.152755 16565 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.60:45945: Network error: Client connection negotiation failed: client connection to 127.2.237.60:45945: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.60.keytab.
Entry for principal kudu/127.2.237.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.60.keytab.
I20260620 23:02:12.157892 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.60:45945
--webserver_interface=127.2.237.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.60.keytab
--principal=kudu/127.2.237.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:41853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:44639,127.2.237.61:39863,127.2.237.60:45945
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:02:12.202116 16565 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } attempt: 1
I20260620 23:02:12.203444 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.202847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:45945 (local address 127.0.0.1:49618)
0620 23:02:12.203098 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:12.203235 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260620 23:02:12.204644 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.204202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:45945 (local address 127.0.0.1:49626)
0620 23:02:12.204369 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:12.204484 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260620 23:02:12.204880 16565 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.60:45945: Network error: Client connection negotiation failed: client connection to 127.2.237.60:45945: connect: Connection refused (error 111)
I20260620 23:02:12.272181 16491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.61" port: 39863 } attempt: 5
I20260620 23:02:12.273217 16565 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } attempt: 2
I20260620 23:02:12.274268 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.273709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:45945 (local address 127.0.0.1:49642)
0620 23:02:12.273982 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:12.274079 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260620 23:02:12.275568 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.275099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:45945 (local address 127.0.0.1:49656)
0620 23:02:12.275314 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:12.275405 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260620 23:02:12.275837 16565 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.60:45945: Network error: Client connection negotiation failed: client connection to 127.2.237.60:45945: connect: Connection refused (error 111)
Jun 20 23:02:12 dist-test-slave-f63b krb5kdc[16411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996531, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for kudu/127.2.237.61@KRBTEST.COM
I20260620 23:02:12.286974 16493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.272969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:39863 (local address 127.0.0.1:59656)
0620 23:02:12.273167 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:12.273191 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:12.273331 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:12.273765 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:12.273771 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:12.274184 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:02:12.274422 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:12.274436 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.275826 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.275834 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:12.276822 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:12.276837 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.277600 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.277606 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:12.277800 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:12.278442 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:02:12.278462 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:02:12.280796 (+ 2334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:02:12.284499 (+ 3703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:12.284504 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:12.284507 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:12.284906 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:12.285321 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:12.285326 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:12.285328 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:12.285480 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:12.286097 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:02:12.286101 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:02:12.286240 (+ 139us) client_negotiation.cc:769] Sending connection context
0620 23:02:12.286366 (+ 126us) client_negotiation.cc:240] Negotiation successful
0620 23:02:12.286713 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260620 23:02:12.287065 16570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.273089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59656 (local address 127.2.237.61:39863)
0620 23:02:12.273327 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:12.273333 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:12.273355 (+ 22us) server_negotiation.cc:407] Connection header received
0620 23:02:12.273512 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:12.273519 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:12.273601 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:12.273716 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:12.274616 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.275651 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.276981 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.277440 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.277594 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:12.281033 (+ 3439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:12.281041 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:12.281046 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:12.281090 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:12.284336 (+ 3246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:12.285035 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:12.285042 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:12.285045 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:12.285195 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:12.285590 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:12.285594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:12.285597 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:12.285969 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:12.286121 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:12.286515 (+ 394us) server_negotiation.cc:299] Negotiation successful
0620 23:02:12.286777 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260620 23:02:12.288286 16491 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:02:12.289556 16493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.289149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:45945 (local address 127.0.0.1:49660)
0620 23:02:12.289330 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:12.289419 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260620 23:02:12.290747 16493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.290372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:45945 (local address 127.0.0.1:49662)
0620 23:02:12.290507 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:12.290634 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260620 23:02:12.291009 16491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.60:45945: Network error: Client connection negotiation failed: client connection to 127.2.237.60:45945: connect: Connection refused (error 111)
I20260620 23:02:12.342281 16491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } attempt: 1
I20260620 23:02:12.343648 16493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.343135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:45945 (local address 127.0.0.1:49664)
0620 23:02:12.343350 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:12.343452 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260620 23:02:12.344720 16493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.344360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:45945 (local address 127.0.0.1:49670)
0620 23:02:12.344515 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:12.344607 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260620 23:02:12.344945 16491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.60:45945: Network error: Client connection negotiation failed: client connection to 127.2.237.60:45945: connect: Connection refused (error 111)
I20260620 23:02:12.367074 16565 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } attempt: 3
I20260620 23:02:12.368366 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.367857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:45945 (local address 127.0.0.1:49674)
0620 23:02:12.368100 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:12.368202 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260620 23:02:12.369468 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.369046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:45945 (local address 127.0.0.1:49688)
0620 23:02:12.369273 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:12.369359 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
W20260620 23:02:12.369690 16565 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.60:45945: Network error: Client connection negotiation failed: client connection to 127.2.237.60:45945: connect: Connection refused (error 111)
I20260620 23:02:12.413172 16491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } attempt: 2
I20260620 23:02:12.414309 16493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.413813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:45945 (local address 127.0.0.1:49704)
0620 23:02:12.414043 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:12.414151 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260620 23:02:12.415398 16493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.415009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:45945 (local address 127.0.0.1:49718)
0620 23:02:12.415159 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:12.415239 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260620 23:02:12.415632 16491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.60:45945: Network error: Client connection negotiation failed: client connection to 127.2.237.60:45945: connect: Connection refused (error 111)
W20260620 23:02:12.510267 16573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:02:12.510679 16573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:02:12.510844 16573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260620 23:02:12.512905 16565 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } attempt: 4
I20260620 23:02:12.514151 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.513638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:45945 (local address 127.0.0.1:49732)
0620 23:02:12.513913 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:12.514000 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260620 23:02:12.515175 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.514853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:45945 (local address 127.0.0.1:49744)
0620 23:02:12.514999 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:12.515075 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260620 23:02:12.515374 16565 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.60:45945: Network error: Client connection negotiation failed: client connection to 127.2.237.60:45945: connect: Connection refused (error 111)
W20260620 23:02:12.518878 16573 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:02:12.519045 16573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:02:12.519153 16573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:02:12.519251 16573 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:02:12.519338 16573 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:02:12.521854 16491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } attempt: 3
I20260620 23:02:12.522980 16493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.522545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:45945 (local address 127.0.0.1:49756)
0620 23:02:12.522743 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:12.522829 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260620 23:02:12.524004 16493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.523622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:45945 (local address 127.0.0.1:49772)
0620 23:02:12.523834 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:12.523904 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:45945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
W20260620 23:02:12.524206 16491 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.60:45945: Network error: Client connection negotiation failed: client connection to 127.2.237.60:45945: connect: Connection refused (error 111)
I20260620 23:02:12.530218 16573 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:41853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:44639,127.2.237.61:39863,127.2.237.60:45945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.60.keytab
--principal=kudu/127.2.237.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.60:45945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.237.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:02:12.531687 16573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:02:12.533028 16573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:02:12.545675 16578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:12.545703 16579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:12.545635 16581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:02:12.546905 16573 server_base.cc:1034] running on GCE node
I20260620 23:02:12.547843 16573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:02:12.549439 16573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:02:12.550684 16573 hybrid_clock.cc:648] HybridClock initialized: now 1781996532550617 us; error 72 us; skew 500 ppm
I20260620 23:02:12.551070 16573 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:02:12 dist-test-slave-f63b krb5kdc[16411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996532, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:02:12.554504 16573 init.cc:377] Logged in from keytab as kudu/127.2.237.60@KRBTEST.COM (short username kudu)
I20260620 23:02:12.556222 16573 webserver.cc:460] Webserver started at http://127.2.237.60:40035/ using document root <none> and password file <none>
I20260620 23:02:12.557150 16573 fs_manager.cc:359] Metadata directory not provided
I20260620 23:02:12.557309 16573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:02:12.557651 16573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:02:12.561753 16573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-2/data/instance:
uuid: "a03befd02ad9454ab8cf6c66e981f7b2"
format_stamp: "Formatted at 2026-06-20 23:02:12 on dist-test-slave-f63b"
server_key: "7333ed0eced732df75d4f7d4b221bc4a"
server_key_iv: "82a353d49cff111019e7a145f0c6295d"
server_key_version: "encryptionkey@0"
I20260620 23:02:12.562880 16573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-2/wal/instance:
uuid: "a03befd02ad9454ab8cf6c66e981f7b2"
format_stamp: "Formatted at 2026-06-20 23:02:12 on dist-test-slave-f63b"
server_key: "7333ed0eced732df75d4f7d4b221bc4a"
server_key_iv: "82a353d49cff111019e7a145f0c6295d"
server_key_version: "encryptionkey@0"
I20260620 23:02:12.569872 16573 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260620 23:02:12.574113 16588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:12.575812 16573 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260620 23:02:12.576054 16573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-2/wal
uuid: "a03befd02ad9454ab8cf6c66e981f7b2"
format_stamp: "Formatted at 2026-06-20 23:02:12 on dist-test-slave-f63b"
server_key: "7333ed0eced732df75d4f7d4b221bc4a"
server_key_iv: "82a353d49cff111019e7a145f0c6295d"
server_key_version: "encryptionkey@0"
I20260620 23:02:12.576310 16573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:02:12.595556 16573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:02:12.600687 16573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:02:12.601023 16573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:02:12.621862 16573 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.60:45945
I20260620 23:02:12.621868 16639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.60:45945 every 8 connection(s)
I20260620 23:02:12.623456 16573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-2/data/info.pb
I20260620 23:02:12.627638 16640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:02:12.632159 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 16573
I20260620 23:02:12.632385 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781996318807807-2996-0/minicluster-data/master-2/wal/instance
I20260620 23:02:12.632851 2996 external_mini_cluster.cc:1428] Setting key 5919c724e4fd18f55ffeddfe980b9660
I20260620 23:02:12.635497 16640 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44639 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:02:12 dist-test-slave-f63b krb5kdc[16411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.60@KRBTEST.COM
Jun 20 23:02:12 dist-test-slave-f63b krb5kdc[16411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996532, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.60@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:02:12.662431 16643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.634991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49774 (local address 127.2.237.60:45945)
0620 23:02:12.635880 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:12.635897 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:12.635949 (+ 52us) server_negotiation.cc:407] Connection header received
0620 23:02:12.637840 (+ 1891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:12.637874 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:12.638445 (+ 571us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:12.638919 (+ 474us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:12.641055 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.643899 (+ 2844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.646973 (+ 3074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.648144 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.648359 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:12.653462 (+ 5103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:12.653472 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:12.653486 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:12.653530 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:12.657161 (+ 3631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:12.657984 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:12.657993 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:12.658005 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:12.658241 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:12.658726 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:12.658732 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:12.658738 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:12.659571 (+ 833us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:12.659905 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:12.660489 (+ 584us) server_negotiation.cc:299] Negotiation successful
0620 23:02:12.660998 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":391,"threads_started":1}
I20260620 23:02:12.665331 16645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.640822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60722 (local address 127.2.237.62:44639)
0620 23:02:12.648181 (+ 7359us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:12.648189 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:12.648210 (+ 21us) server_negotiation.cc:407] Connection header received
0620 23:02:12.648278 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:12.648283 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:12.648380 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:12.648496 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:12.650174 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.650943 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.652757 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.653190 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.653354 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:12.658107 (+ 4753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:12.658118 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:12.658125 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:12.658171 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:12.661806 (+ 3635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:12.662815 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:12.662823 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:12.662827 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:12.663051 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:12.663523 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:12.663528 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:12.663531 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:12.663816 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:12.664077 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:12.664603 (+ 526us) server_negotiation.cc:299] Negotiation successful
0620 23:02:12.664857 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7226,"thread_start_us":7054,"threads_started":1}
I20260620 23:02:12.665356 16644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.640197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:44639 (local address 127.0.0.1:60722)
0620 23:02:12.647501 (+ 7304us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:12.647587 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:12.647824 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:12.648612 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:12.648621 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:12.649574 (+ 953us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:02:12.649968 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:12.649983 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.651113 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.651118 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:12.652576 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:12.652589 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.653463 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.653470 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:12.653687 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:12.654424 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:02:12.654454 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:02:12.657161 (+ 2707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:02:12.661967 (+ 4806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:12.661979 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:12.661985 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:12.662648 (+ 663us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:12.663184 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:12.663193 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:12.663200 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:12.663396 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:12.664030 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:02:12.664039 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:02:12.664250 (+ 211us) client_negotiation.cc:769] Sending connection context
0620 23:02:12.664464 (+ 214us) client_negotiation.cc:240] Negotiation successful
0620 23:02:12.664828 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6903,"thread_start_us":6684,"threads_started":1}
I20260620 23:02:12.667398 16640 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.61" port: 39863 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:02:12 dist-test-slave-f63b krb5kdc[16411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996532, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.60@KRBTEST.COM for kudu/127.2.237.61@KRBTEST.COM
Jun 20 23:02:12 dist-test-slave-f63b krb5kdc[16411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:02:12.682281 16644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.668302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:39863 (local address 127.0.0.1:59672)
0620 23:02:12.668520 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:12.668534 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:12.668637 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:12.668996 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:12.668999 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:12.669369 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:02:12.669513 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:12.669521 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.671079 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.671085 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:12.671985 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:12.671998 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.672747 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.672752 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:12.672950 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:12.673552 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:02:12.673570 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:02:12.675931 (+ 2361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:02:12.679438 (+ 3507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:12.679445 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:12.679449 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:12.679892 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:12.680384 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:12.680390 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:12.680394 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:12.680622 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:12.681164 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:02:12.681170 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:02:12.681405 (+ 235us) client_negotiation.cc:769] Sending connection context
0620 23:02:12.681568 (+ 163us) client_negotiation.cc:240] Negotiation successful
0620 23:02:12.681896 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135}
I20260620 23:02:12.682310 16570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.668288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59672 (local address 127.2.237.61:39863)
0620 23:02:12.668514 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:12.668520 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:12.668601 (+ 81us) server_negotiation.cc:407] Connection header received
0620 23:02:12.668759 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:12.668765 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:12.668851 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:12.668969 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:12.669686 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.670931 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.672151 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.672587 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.672731 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:12.676174 (+ 3443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:12.676179 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:12.676182 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:12.676221 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:12.679270 (+ 3049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:12.680059 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:12.680063 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:12.680066 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:12.680252 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:12.680734 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:12.680739 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:12.680743 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:12.681027 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:12.681192 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:12.681683 (+ 491us) server_negotiation.cc:299] Negotiation successful
0620 23:02:12.681911 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260620 23:02:12.683396 16645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.665804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60730 (local address 127.2.237.62:44639)
0620 23:02:12.666055 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:12.666063 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:12.666082 (+ 19us) server_negotiation.cc:407] Connection header received
0620 23:02:12.666142 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:12.666149 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:12.666247 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:12.666353 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:12.668195 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.669386 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.671515 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.671981 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.672137 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:12.677303 (+ 5166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:12.677309 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:12.677313 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:12.677351 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:12.680297 (+ 2946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:12.681162 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:12.681166 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:12.681169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:12.681360 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:12.681859 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:12.681863 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:12.681865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:12.682144 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:12.682449 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:12.682781 (+ 332us) server_negotiation.cc:299] Negotiation successful
0620 23:02:12.683071 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260620 23:02:12.684036 16640 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:02:12 dist-test-slave-f63b krb5kdc[16411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996532, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.60@KRBTEST.COM for kudu/127.2.237.60@KRBTEST.COM
Jun 20 23:02:12 dist-test-slave-f63b krb5kdc[16411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.61@KRBTEST.COM
I20260620 23:02:12.699379 16643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.685114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49786 (local address 127.2.237.60:45945)
0620 23:02:12.685259 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:12.685263 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:12.685277 (+ 14us) server_negotiation.cc:407] Connection header received
0620 23:02:12.685318 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:12.685322 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:12.685375 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:12.685451 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:12.686395 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.687196 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.688527 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.688970 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.689103 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:12.692935 (+ 3832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:12.692940 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:12.692943 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:12.692976 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:12.696344 (+ 3368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:12.697160 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:12.697164 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:12.697166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:12.697290 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:12.697691 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:12.697694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:12.697697 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:12.698151 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:12.698314 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:12.698728 (+ 414us) server_negotiation.cc:299] Negotiation successful
0620 23:02:12.698965 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 23:02:12.699589 16644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.684853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:45945 (local address 127.0.0.1:49786)
0620 23:02:12.685030 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:12.685051 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:12.685170 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:12.685516 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:12.685521 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:12.686058 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:02:12.686270 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:12.686283 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.687331 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.687337 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:12.688357 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:12.688369 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.689150 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.689156 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:12.689347 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:12.689984 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:02:12.690014 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:02:12.692708 (+ 2694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:02:12.696480 (+ 3772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:12.696488 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:12.696493 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:12.697024 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:12.697405 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:12.697410 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:12.697414 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:12.697577 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:12.698287 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:02:12.698292 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:02:12.698439 (+ 147us) client_negotiation.cc:769] Sending connection context
0620 23:02:12.698652 (+ 213us) client_negotiation.cc:240] Negotiation successful
0620 23:02:12.699124 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":153}
I20260620 23:02:12.701452 16491 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } attempt: 4
I20260620 23:02:12.704577 16570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.685539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59678 (local address 127.2.237.61:39863)
0620 23:02:12.685701 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:12.685705 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:12.685856 (+ 151us) server_negotiation.cc:407] Connection header received
0620 23:02:12.686013 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:12.686017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:12.686074 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:12.686148 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:12.687653 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.688364 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.689988 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.690424 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.690532 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:12.696161 (+ 5629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:12.696167 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:12.696170 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:12.696211 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:12.699406 (+ 3195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:12.700834 (+ 1428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:12.700842 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:12.700845 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:12.701083 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:12.701648 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:12.701657 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:12.701662 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:12.702105 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:12.702417 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:12.703803 (+ 1386us) server_negotiation.cc:299] Negotiation successful
0620 23:02:12.704123 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 23:02:12.707922 16640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2: Bootstrap starting.
I20260620 23:02:12.711035 2996 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:02:12.712544 16640 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2: Neither blocks nor log segments found. Creating new log.
I20260620 23:02:12.714003 16640 log.cc:824] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2: Log is configured to *not* fsync() on all Append() calls
Jun 20 23:02:12 dist-test-slave-f63b krb5kdc[16411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996531, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for kudu/127.2.237.60@KRBTEST.COM
I20260620 23:02:12.718283 16640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2: No bootstrap required, opened a new log
I20260620 23:02:12.721609 16493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.702429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:45945 (local address 127.0.0.1:49792)
0620 23:02:12.702777 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:12.702797 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:12.702964 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:12.706700 (+ 3736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:12.706706 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:12.707323 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:02:12.707508 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:12.707532 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.709584 (+ 2052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.709593 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:12.711178 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:12.711201 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.712086 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.712092 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:12.712299 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:12.712984 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:02:12.713003 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:02:12.715445 (+ 2442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:02:12.718840 (+ 3395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:12.718844 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:12.718848 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:12.719228 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:12.719715 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:12.719718 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:12.719721 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:12.719859 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:12.720540 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:02:12.720545 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:02:12.720708 (+ 163us) client_negotiation.cc:769] Sending connection context
0620 23:02:12.720850 (+ 142us) client_negotiation.cc:240] Negotiation successful
0620 23:02:12.721190 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167}
I20260620 23:02:12.721616 16643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.705613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49792 (local address 127.2.237.60:45945)
0620 23:02:12.706173 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:12.706179 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:12.706199 (+ 20us) server_negotiation.cc:407] Connection header received
0620 23:02:12.706287 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:12.706294 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:12.706437 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:12.707674 (+ 1237us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:12.707784 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.709318 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.711450 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.711925 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.712066 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:12.715688 (+ 3622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:12.715693 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:12.715696 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:12.715726 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:12.718646 (+ 2920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:12.719369 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:12.719377 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:12.719383 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:12.719566 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:12.719988 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:12.719994 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:12.719997 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:12.720398 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:12.720597 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:12.720988 (+ 391us) server_negotiation.cc:299] Negotiation successful
0620 23:02:12.721235 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333}
I20260620 23:02:12.731197 16491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfb1d92e25794bbaaec541472c7682a1: Bootstrap starting.
I20260620 23:02:12.737046 16491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfb1d92e25794bbaaec541472c7682a1: Neither blocks nor log segments found. Creating new log.
I20260620 23:02:12.739042 16491 log.cc:824] T 00000000000000000000000000000000 P bfb1d92e25794bbaaec541472c7682a1: Log is configured to *not* fsync() on all Append() calls
I20260620 23:02:12.739014 16640 raft_consensus.cc:348] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb1d92e25794bbaaec541472c7682a1" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44639 } } peers { permanent_uuid: "5e34876e039e4944ba27d633b7a071ce" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 39863 } } peers { permanent_uuid: "a03befd02ad9454ab8cf6c66e981f7b2" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } }
I20260620 23:02:12.739509 16640 raft_consensus.cc:374] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:02:12.739691 16640 raft_consensus.cc:729] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a03befd02ad9454ab8cf6c66e981f7b2, State: Initialized, Role: FOLLOWER
I20260620 23:02:12.743745 16491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfb1d92e25794bbaaec541472c7682a1: No bootstrap required, opened a new log
I20260620 23:02:12.747776 16640 consensus_queue.cc:260] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb1d92e25794bbaaec541472c7682a1" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44639 } } peers { permanent_uuid: "5e34876e039e4944ba27d633b7a071ce" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 39863 } } peers { permanent_uuid: "a03befd02ad9454ab8cf6c66e981f7b2" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } }
I20260620 23:02:12.748906 16656 sys_catalog.cc:455] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb1d92e25794bbaaec541472c7682a1" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44639 } } peers { permanent_uuid: "5e34876e039e4944ba27d633b7a071ce" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 39863 } } peers { permanent_uuid: "a03befd02ad9454ab8cf6c66e981f7b2" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } } }
I20260620 23:02:12.749356 16656 sys_catalog.cc:458] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:02:12.752250 16640 sys_catalog.cc:564] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:02:12.755879 16570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.722759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59686 (local address 127.2.237.61:39863)
0620 23:02:12.722941 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:12.722946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:12.734854 (+ 11908us) server_negotiation.cc:407] Connection header received
0620 23:02:12.735393 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:12.735400 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:12.735488 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:12.735558 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:12.738051 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.739227 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.742435 (+ 3208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.743220 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.743336 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:12.747462 (+ 4126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:12.747469 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:12.747473 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:12.747521 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:12.751425 (+ 3904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:12.752488 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:12.752494 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:12.752498 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:12.752658 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:12.753219 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:12.753224 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:12.753229 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:12.753600 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:12.753867 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:12.755270 (+ 1403us) server_negotiation.cc:299] Negotiation successful
0620 23:02:12.755527 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260620 23:02:12.755937 16491 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfb1d92e25794bbaaec541472c7682a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb1d92e25794bbaaec541472c7682a1" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44639 } } peers { permanent_uuid: "5e34876e039e4944ba27d633b7a071ce" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 39863 } } peers { permanent_uuid: "a03befd02ad9454ab8cf6c66e981f7b2" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } }
I20260620 23:02:12.756371 16491 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfb1d92e25794bbaaec541472c7682a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:02:12.756491 16491 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfb1d92e25794bbaaec541472c7682a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfb1d92e25794bbaaec541472c7682a1, State: Initialized, Role: FOLLOWER
I20260620 23:02:12.757421 16491 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfb1d92e25794bbaaec541472c7682a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb1d92e25794bbaaec541472c7682a1" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44639 } } peers { permanent_uuid: "5e34876e039e4944ba27d633b7a071ce" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 39863 } } peers { permanent_uuid: "a03befd02ad9454ab8cf6c66e981f7b2" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } }
I20260620 23:02:12.764397 16643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.721759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49798 (local address 127.2.237.60:45945)
0620 23:02:12.722328 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:12.722335 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:12.740708 (+ 18373us) server_negotiation.cc:407] Connection header received
0620 23:02:12.741018 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:12.741024 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:12.741112 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:12.741189 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:12.743734 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.744865 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.748504 (+ 3639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.749265 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.749387 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:12.752887 (+ 3500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:12.752894 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:12.752899 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:12.752958 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:12.759576 (+ 6618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:12.760900 (+ 1324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:12.760909 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:12.760914 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:12.761137 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:12.761749 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:12.761756 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:12.761763 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:12.762297 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:12.762691 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:12.763423 (+ 732us) server_negotiation.cc:299] Negotiation successful
0620 23:02:12.763730 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":5886,"threads_started":1}
I20260620 23:02:12.766108 16658 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfb1d92e25794bbaaec541472c7682a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb1d92e25794bbaaec541472c7682a1" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44639 } } peers { permanent_uuid: "5e34876e039e4944ba27d633b7a071ce" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 39863 } } peers { permanent_uuid: "a03befd02ad9454ab8cf6c66e981f7b2" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } } }
I20260620 23:02:12.766664 16658 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfb1d92e25794bbaaec541472c7682a1 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:02:12.767081 16491 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfb1d92e25794bbaaec541472c7682a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:02:12.768966 16645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.722777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60740 (local address 127.2.237.62:44639)
0620 23:02:12.722953 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:12.722959 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:12.727401 (+ 4442us) server_negotiation.cc:407] Connection header received
0620 23:02:12.730413 (+ 3012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:12.730424 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:12.730547 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:12.730673 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:12.740994 (+ 10321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.742192 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.744578 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.745222 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.745323 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:12.758134 (+ 12811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:12.758145 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:12.758153 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:12.758215 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:12.763548 (+ 5333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:12.765072 (+ 1524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:12.765079 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:12.765082 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:12.765895 (+ 813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:12.766406 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:12.766413 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:12.766416 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:12.766925 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:12.767307 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:12.768079 (+ 772us) server_negotiation.cc:299] Negotiation successful
0620 23:02:12.768464 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":61}
W20260620 23:02:12.783001 16674 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:02:12.783353 16674 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:02:12.775799 16573 master_runner.cc:186] Error getting master registration for 127.2.237.62:44639: OK, instance_id { permanent_uuid: "bfb1d92e25794bbaaec541472c7682a1" instance_seqno: 1781996531551015 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 23:02:12.787009 16573 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 23:02:12.789844 16445 tablet.cc:2369] T 00000000000000000000000000000000 P bfb1d92e25794bbaaec541472c7682a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:02:12.793576 16680 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bfb1d92e25794bbaaec541472c7682a1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:02:12.793838 16680 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:02:12.791293 16424 master_runner.cc:186] Error getting master registration for 127.2.237.61:39863: OK, instance_id { permanent_uuid: "5e34876e039e4944ba27d633b7a071ce" instance_seqno: 1781996532084117 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 23:02:12.802999 16424 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 23:02:12.814616 16565 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } attempt: 5
Jun 20 23:02:12 dist-test-slave-f63b krb5kdc[16411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996532, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.61@KRBTEST.COM for kudu/127.2.237.60@KRBTEST.COM
I20260620 23:02:12.835117 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.815343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:45945 (local address 127.0.0.1:49812)
0620 23:02:12.815635 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:12.815655 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:12.815788 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:12.816190 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:12.816197 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:12.816800 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:02:12.817031 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:12.817045 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.818561 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.818595 (+ 34us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:12.820054 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:12.820071 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.821124 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.821131 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:12.821367 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:12.822125 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:02:12.822152 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:02:12.827316 (+ 5164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:02:12.831889 (+ 4573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:12.831896 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:12.831901 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:12.832531 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:12.833006 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:12.833012 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:12.833016 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:12.833208 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:12.833929 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:02:12.833935 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:02:12.834131 (+ 196us) client_negotiation.cc:769] Sending connection context
0620 23:02:12.834308 (+ 177us) client_negotiation.cc:240] Negotiation successful
0620 23:02:12.834684 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170}
I20260620 23:02:12.835165 16643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:12.815426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49812 (local address 127.2.237.60:45945)
0620 23:02:12.815616 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:12.815623 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:12.815739 (+ 116us) server_negotiation.cc:407] Connection header received
0620 23:02:12.815920 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:12.815926 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:12.816018 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:12.816141 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:12.817217 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.818367 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.820228 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:12.820931 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:12.821073 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:12.827587 (+ 6514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:12.827595 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:12.827612 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:12.827660 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:12.831686 (+ 4026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:12.832682 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:12.832688 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:12.832692 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:12.832869 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:12.833343 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:12.833349 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:12.833353 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:12.833782 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:12.833967 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:12.834424 (+ 457us) server_negotiation.cc:299] Negotiation successful
0620 23:02:12.834731 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260620 23:02:12.842093 16565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e34876e039e4944ba27d633b7a071ce: Bootstrap starting.
I20260620 23:02:12.846494 16565 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e34876e039e4944ba27d633b7a071ce: Neither blocks nor log segments found. Creating new log.
I20260620 23:02:12.848071 16565 log.cc:824] T 00000000000000000000000000000000 P 5e34876e039e4944ba27d633b7a071ce: Log is configured to *not* fsync() on all Append() calls
W20260620 23:02:12.853534 16594 tablet.cc:2369] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:02:12.855943 16565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e34876e039e4944ba27d633b7a071ce: No bootstrap required, opened a new log
I20260620 23:02:12.868042 16565 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e34876e039e4944ba27d633b7a071ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb1d92e25794bbaaec541472c7682a1" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44639 } } peers { permanent_uuid: "5e34876e039e4944ba27d633b7a071ce" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 39863 } } peers { permanent_uuid: "a03befd02ad9454ab8cf6c66e981f7b2" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } }
I20260620 23:02:12.868563 16565 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e34876e039e4944ba27d633b7a071ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:02:12.868683 16565 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e34876e039e4944ba27d633b7a071ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e34876e039e4944ba27d633b7a071ce, State: Initialized, Role: FOLLOWER
I20260620 23:02:12.869433 16565 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e34876e039e4944ba27d633b7a071ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb1d92e25794bbaaec541472c7682a1" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44639 } } peers { permanent_uuid: "5e34876e039e4944ba27d633b7a071ce" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 39863 } } peers { permanent_uuid: "a03befd02ad9454ab8cf6c66e981f7b2" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } }
I20260620 23:02:12.872336 16683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e34876e039e4944ba27d633b7a071ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb1d92e25794bbaaec541472c7682a1" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44639 } } peers { permanent_uuid: "5e34876e039e4944ba27d633b7a071ce" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 39863 } } peers { permanent_uuid: "a03befd02ad9454ab8cf6c66e981f7b2" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } } }
I20260620 23:02:12.873039 16683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e34876e039e4944ba27d633b7a071ce [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:02:12.873845 16565 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e34876e039e4944ba27d633b7a071ce [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:02:12.891309 16498 master_runner.cc:263] No leader master found from master 5e34876e039e4944ba27d633b7a071ce
I20260620 23:02:12.891516 16498 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260620 23:02:12.897786 16694 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5e34876e039e4944ba27d633b7a071ce: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:02:12.898082 16694 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260620 23:02:13.073103 16519 tablet.cc:2369] T 00000000000000000000000000000000 P 5e34876e039e4944ba27d633b7a071ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:02:13.078965 16656 raft_consensus.cc:482] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:02:13.079344 16656 raft_consensus.cc:504] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb1d92e25794bbaaec541472c7682a1" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44639 } } peers { permanent_uuid: "5e34876e039e4944ba27d633b7a071ce" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 39863 } } peers { permanent_uuid: "a03befd02ad9454ab8cf6c66e981f7b2" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } }
I20260620 23:02:13.081198 16656 leader_election.cc:288] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfb1d92e25794bbaaec541472c7682a1 (127.2.237.62:44639), 5e34876e039e4944ba27d633b7a071ce (127.2.237.61:39863)
I20260620 23:02:13.083840 16466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a03befd02ad9454ab8cf6c66e981f7b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfb1d92e25794bbaaec541472c7682a1" is_pre_election: true
I20260620 23:02:13.084558 16466 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bfb1d92e25794bbaaec541472c7682a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a03befd02ad9454ab8cf6c66e981f7b2 in term 0.
I20260620 23:02:13.085551 16589 leader_election.cc:302] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a03befd02ad9454ab8cf6c66e981f7b2, bfb1d92e25794bbaaec541472c7682a1; no voters:
I20260620 23:02:13.086215 16656 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:02:13.086375 16656 raft_consensus.cc:482] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:02:13.086472 16656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:02:13.090072 16656 raft_consensus.cc:504] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb1d92e25794bbaaec541472c7682a1" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44639 } } peers { permanent_uuid: "5e34876e039e4944ba27d633b7a071ce" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 39863 } } peers { permanent_uuid: "a03befd02ad9454ab8cf6c66e981f7b2" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } }
I20260620 23:02:13.091956 16540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a03befd02ad9454ab8cf6c66e981f7b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e34876e039e4944ba27d633b7a071ce" is_pre_election: true
I20260620 23:02:13.092731 16540 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5e34876e039e4944ba27d633b7a071ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a03befd02ad9454ab8cf6c66e981f7b2 in term 0.
I20260620 23:02:13.100028 16466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a03befd02ad9454ab8cf6c66e981f7b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfb1d92e25794bbaaec541472c7682a1"
I20260620 23:02:13.100328 16466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfb1d92e25794bbaaec541472c7682a1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:02:13.111092 16466 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bfb1d92e25794bbaaec541472c7682a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a03befd02ad9454ab8cf6c66e981f7b2 in term 1.
I20260620 23:02:13.112236 16540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a03befd02ad9454ab8cf6c66e981f7b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e34876e039e4944ba27d633b7a071ce"
I20260620 23:02:13.112507 16540 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e34876e039e4944ba27d633b7a071ce [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:02:13.126957 16540 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5e34876e039e4944ba27d633b7a071ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a03befd02ad9454ab8cf6c66e981f7b2 in term 1.
I20260620 23:02:13.127777 16591 leader_election.cc:302] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e34876e039e4944ba27d633b7a071ce, a03befd02ad9454ab8cf6c66e981f7b2; no voters:
I20260620 23:02:13.130970 16696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:02:13.131361 16656 leader_election.cc:288] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2 [CANDIDATE]: Term 1 election: Requested vote from peers bfb1d92e25794bbaaec541472c7682a1 (127.2.237.62:44639), 5e34876e039e4944ba27d633b7a071ce (127.2.237.61:39863)
I20260620 23:02:13.142016 16696 raft_consensus.cc:686] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2 [term 1 LEADER]: Becoming Leader. State: Replica: a03befd02ad9454ab8cf6c66e981f7b2, State: Running, Role: LEADER
I20260620 23:02:13.142792 16696 consensus_queue.cc:237] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb1d92e25794bbaaec541472c7682a1" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44639 } } peers { permanent_uuid: "5e34876e039e4944ba27d633b7a071ce" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 39863 } } peers { permanent_uuid: "a03befd02ad9454ab8cf6c66e981f7b2" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } }
I20260620 23:02:13.155416 16656 sys_catalog.cc:455] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a03befd02ad9454ab8cf6c66e981f7b2. Latest consensus state: current_term: 1 leader_uuid: "a03befd02ad9454ab8cf6c66e981f7b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb1d92e25794bbaaec541472c7682a1" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44639 } } peers { permanent_uuid: "5e34876e039e4944ba27d633b7a071ce" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 39863 } } peers { permanent_uuid: "a03befd02ad9454ab8cf6c66e981f7b2" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } } }
I20260620 23:02:13.155829 16656 sys_catalog.cc:458] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2 [sys.catalog]: This master's current role is: LEADER
I20260620 23:02:13.163329 16699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:02:13.166039 16699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:02:13.180991 16540 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5e34876e039e4944ba27d633b7a071ce [term 1 FOLLOWER]: Refusing update from remote peer a03befd02ad9454ab8cf6c66e981f7b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:02:13.182423 16656 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e34876e039e4944ba27d633b7a071ce" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 39863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:02:13.201227 16466 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bfb1d92e25794bbaaec541472c7682a1 [term 1 FOLLOWER]: Refusing update from remote peer a03befd02ad9454ab8cf6c66e981f7b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:02:13.213016 16656 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb1d92e25794bbaaec541472c7682a1" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:02:13.230842 16683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e34876e039e4944ba27d633b7a071ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader a03befd02ad9454ab8cf6c66e981f7b2. Latest consensus state: current_term: 1 leader_uuid: "a03befd02ad9454ab8cf6c66e981f7b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb1d92e25794bbaaec541472c7682a1" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44639 } } peers { permanent_uuid: "5e34876e039e4944ba27d633b7a071ce" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 39863 } } peers { permanent_uuid: "a03befd02ad9454ab8cf6c66e981f7b2" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } } }
I20260620 23:02:13.231173 16683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e34876e039e4944ba27d633b7a071ce [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:02:13.249092 16658 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfb1d92e25794bbaaec541472c7682a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a03befd02ad9454ab8cf6c66e981f7b2. Latest consensus state: current_term: 1 leader_uuid: "a03befd02ad9454ab8cf6c66e981f7b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb1d92e25794bbaaec541472c7682a1" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44639 } } peers { permanent_uuid: "5e34876e039e4944ba27d633b7a071ce" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 39863 } } peers { permanent_uuid: "a03befd02ad9454ab8cf6c66e981f7b2" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } } }
I20260620 23:02:13.249480 16658 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfb1d92e25794bbaaec541472c7682a1 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:02:13.253034 16702 mvcc.cc:205] Tried to move back new op lower bound from 7299057799880101888 to 7299057799759032320. Current Snapshot: MvccSnapshot[applied={T|T < 7299057799880101888}]
I20260620 23:02:13.264245 16658 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfb1d92e25794bbaaec541472c7682a1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a03befd02ad9454ab8cf6c66e981f7b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb1d92e25794bbaaec541472c7682a1" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44639 } } peers { permanent_uuid: "5e34876e039e4944ba27d633b7a071ce" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 39863 } } peers { permanent_uuid: "a03befd02ad9454ab8cf6c66e981f7b2" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } } }
I20260620 23:02:13.264561 16658 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfb1d92e25794bbaaec541472c7682a1 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:02:13.269140 16699 catalog_manager.cc:1347] Generated new cluster ID: fab1cf7232044128a6ac47201e54b2ce
I20260620 23:02:13.269291 16699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:02:13.269833 16683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e34876e039e4944ba27d633b7a071ce [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a03befd02ad9454ab8cf6c66e981f7b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb1d92e25794bbaaec541472c7682a1" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44639 } } peers { permanent_uuid: "5e34876e039e4944ba27d633b7a071ce" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 39863 } } peers { permanent_uuid: "a03befd02ad9454ab8cf6c66e981f7b2" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } } }
I20260620 23:02:13.270179 16683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e34876e039e4944ba27d633b7a071ce [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:02:13.271785 16696 sys_catalog.cc:455] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a03befd02ad9454ab8cf6c66e981f7b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb1d92e25794bbaaec541472c7682a1" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44639 } } peers { permanent_uuid: "5e34876e039e4944ba27d633b7a071ce" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 39863 } } peers { permanent_uuid: "a03befd02ad9454ab8cf6c66e981f7b2" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } } }
I20260620 23:02:13.272140 16656 sys_catalog.cc:455] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a03befd02ad9454ab8cf6c66e981f7b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfb1d92e25794bbaaec541472c7682a1" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44639 } } peers { permanent_uuid: "5e34876e039e4944ba27d633b7a071ce" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 39863 } } peers { permanent_uuid: "a03befd02ad9454ab8cf6c66e981f7b2" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 45945 } } }
I20260620 23:02:13.272390 16656 sys_catalog.cc:458] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2 [sys.catalog]: This master's current role is: LEADER
I20260620 23:02:13.273876 16696 sys_catalog.cc:458] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2 [sys.catalog]: This master's current role is: LEADER
I20260620 23:02:13.325009 16699 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:02:13.327211 16699 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:02:13.352739 16699 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a03befd02ad9454ab8cf6c66e981f7b2: Generated new TSK 0
I20260620 23:02:13.354417 16699 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:02:13.795823 16680 catalog_manager.cc:1259] Loaded cluster ID: fab1cf7232044128a6ac47201e54b2ce
I20260620 23:02:13.795940 16680 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bfb1d92e25794bbaaec541472c7682a1: loading cluster ID for follower catalog manager: success
I20260620 23:02:13.798482 16680 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bfb1d92e25794bbaaec541472c7682a1: acquiring CA information for follower catalog manager: success
I20260620 23:02:13.800359 16680 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bfb1d92e25794bbaaec541472c7682a1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 23:02:13.900175 16694 catalog_manager.cc:1259] Loaded cluster ID: fab1cf7232044128a6ac47201e54b2ce
I20260620 23:02:13.900285 16694 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5e34876e039e4944ba27d633b7a071ce: loading cluster ID for follower catalog manager: success
I20260620 23:02:13.902772 16694 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5e34876e039e4944ba27d633b7a071ce: acquiring CA information for follower catalog manager: success
I20260620 23:02:13.904670 16694 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5e34876e039e4944ba27d633b7a071ce: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 23:02:13.929909 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 16424
I20260620 23:02:13.931262 16486 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:02:14.046171 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 16424
I20260620 23:02:14.067639 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 16498
I20260620 23:02:14.068837 16560 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:02:14.178431 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 16498
I20260620 23:02:14.195811 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 16573
I20260620 23:02:14.196918 16635 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:02:14.309305 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 16573
2026-06-20T23:02:14Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4583 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:02:14 dist-test-slave-f63b krb5kdc[16717](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:02:14 dist-test-slave-f63b krb5kdc[16717](info): set up 2 sockets
Jun 20 23:02:14 dist-test-slave-f63b krb5kdc[16717](info): commencing operation
krb5kdc: starting...
W20260620 23:02:15.431206 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.006s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 23:02:15 dist-test-slave-f63b krb5kdc[16717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:02:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:02:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
I20260620 23:02:15.674690 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:40811
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:33081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:40811
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:02:16.016299 16730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:02:16.016702 16730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:02:16.016858 16730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:02:16.024415 16730 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:02:16.024597 16730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:02:16.024711 16730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:02:16.024816 16730 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:02:16.024891 16730 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:02:16.035285 16730 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:33081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:40811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:40811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:02:16.036840 16730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:02:16.038203 16730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:02:16.050506 16735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:16.050537 16736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:16.050459 16738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:02:16.051805 16730 server_base.cc:1034] running on GCE node
I20260620 23:02:16.052650 16730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:02:16.054229 16730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:02:16.055469 16730 hybrid_clock.cc:648] HybridClock initialized: now 1781996536055417 us; error 48 us; skew 500 ppm
I20260620 23:02:16.055852 16730 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:02:16 dist-test-slave-f63b krb5kdc[16717](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996536, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:02:16.059152 16730 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 23:02:16.060729 16730 webserver.cc:460] Webserver started at http://127.2.237.62:38043/ using document root <none> and password file <none>
I20260620 23:02:16.061487 16730 fs_manager.cc:359] Metadata directory not provided
I20260620 23:02:16.061645 16730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:02:16.061970 16730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:02:16.065222 16730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "606b7c6fac904807ba2124551310b6c8"
format_stamp: "Formatted at 2026-06-20 23:02:16 on dist-test-slave-f63b"
server_key: "0ae97b736ae5986510f8e61d93b1d5e1"
server_key_iv: "5c9e973df6e912c14b9236af28096550"
server_key_version: "encryptionkey@0"
I20260620 23:02:16.066036 16730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "606b7c6fac904807ba2124551310b6c8"
format_stamp: "Formatted at 2026-06-20 23:02:16 on dist-test-slave-f63b"
server_key: "0ae97b736ae5986510f8e61d93b1d5e1"
server_key_iv: "5c9e973df6e912c14b9236af28096550"
server_key_version: "encryptionkey@0"
I20260620 23:02:16.071765 16730 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260620 23:02:16.075594 16745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:16.077037 16730 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260620 23:02:16.077241 16730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "606b7c6fac904807ba2124551310b6c8"
format_stamp: "Formatted at 2026-06-20 23:02:16 on dist-test-slave-f63b"
server_key: "0ae97b736ae5986510f8e61d93b1d5e1"
server_key_iv: "5c9e973df6e912c14b9236af28096550"
server_key_version: "encryptionkey@0"
I20260620 23:02:16.077461 16730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:02:16.101625 16730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:02:16.105991 16730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:02:16.106307 16730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:02:16.125828 16730 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:40811
I20260620 23:02:16.125847 16796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:40811 every 8 connection(s)
I20260620 23:02:16.127441 16730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:02:16.128764 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 16730
I20260620 23:02:16.129004 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:02:16.129822 2996 external_mini_cluster.cc:1428] Setting key 20c3515940cfb24f3ad2cc37b99bffcb
I20260620 23:02:16.132673 16797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 20 23:02:16 dist-test-slave-f63b krb5kdc[16717](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:02:16.146456 16797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 606b7c6fac904807ba2124551310b6c8: Bootstrap starting.
I20260620 23:02:16.150913 16797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 606b7c6fac904807ba2124551310b6c8: Neither blocks nor log segments found. Creating new log.
I20260620 23:02:16.152153 16797 log.cc:824] T 00000000000000000000000000000000 P 606b7c6fac904807ba2124551310b6c8: Log is configured to *not* fsync() on all Append() calls
I20260620 23:02:16.155804 16797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 606b7c6fac904807ba2124551310b6c8: No bootstrap required, opened a new log
I20260620 23:02:16.155956 16800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:16.131872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59730 (local address 127.2.237.62:40811)
0620 23:02:16.132688 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:16.132700 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:16.132770 (+ 70us) server_negotiation.cc:407] Connection header received
0620 23:02:16.134486 (+ 1716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:16.134510 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:16.134978 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:16.135348 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:16.137259 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:16.138752 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:16.140672 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:16.141419 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:16.141570 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:16.146743 (+ 5173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:16.146755 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:16.146777 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:16.146839 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:16.150914 (+ 4075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:16.151740 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:16.151749 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:16.151758 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:16.151991 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:16.152500 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:16.152505 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:16.152509 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:16.153247 (+ 738us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:16.153470 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:16.154051 (+ 581us) server_negotiation.cc:299] Negotiation successful
0620 23:02:16.154496 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":386,"threads_started":1}
I20260620 23:02:16.160285 2996 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:02:16.164667 16797 raft_consensus.cc:348] T 00000000000000000000000000000000 P 606b7c6fac904807ba2124551310b6c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "606b7c6fac904807ba2124551310b6c8" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 40811 } }
I20260620 23:02:16.165087 16797 raft_consensus.cc:374] T 00000000000000000000000000000000 P 606b7c6fac904807ba2124551310b6c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:02:16.165198 16797 raft_consensus.cc:729] T 00000000000000000000000000000000 P 606b7c6fac904807ba2124551310b6c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 606b7c6fac904807ba2124551310b6c8, State: Initialized, Role: FOLLOWER
I20260620 23:02:16.165843 16797 consensus_queue.cc:260] T 00000000000000000000000000000000 P 606b7c6fac904807ba2124551310b6c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "606b7c6fac904807ba2124551310b6c8" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 40811 } }
I20260620 23:02:16.166096 16797 raft_consensus.cc:388] T 00000000000000000000000000000000 P 606b7c6fac904807ba2124551310b6c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:02:16.166231 16797 raft_consensus.cc:482] T 00000000000000000000000000000000 P 606b7c6fac904807ba2124551310b6c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:02:16.166409 16797 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 606b7c6fac904807ba2124551310b6c8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:02:16.168718 16797 raft_consensus.cc:504] T 00000000000000000000000000000000 P 606b7c6fac904807ba2124551310b6c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "606b7c6fac904807ba2124551310b6c8" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 40811 } }
I20260620 23:02:16.169251 16797 leader_election.cc:302] T 00000000000000000000000000000000 P 606b7c6fac904807ba2124551310b6c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 606b7c6fac904807ba2124551310b6c8; no voters:
I20260620 23:02:16.169838 16797 leader_election.cc:288] T 00000000000000000000000000000000 P 606b7c6fac904807ba2124551310b6c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:02:16.170038 16802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 606b7c6fac904807ba2124551310b6c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:02:16.170774 16802 raft_consensus.cc:686] T 00000000000000000000000000000000 P 606b7c6fac904807ba2124551310b6c8 [term 1 LEADER]: Becoming Leader. State: Replica: 606b7c6fac904807ba2124551310b6c8, State: Running, Role: LEADER
I20260620 23:02:16.171366 16802 consensus_queue.cc:237] T 00000000000000000000000000000000 P 606b7c6fac904807ba2124551310b6c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "606b7c6fac904807ba2124551310b6c8" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 40811 } }
I20260620 23:02:16.172029 16797 sys_catalog.cc:564] T 00000000000000000000000000000000 P 606b7c6fac904807ba2124551310b6c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:02:16.174651 16804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 606b7c6fac904807ba2124551310b6c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 606b7c6fac904807ba2124551310b6c8. Latest consensus state: current_term: 1 leader_uuid: "606b7c6fac904807ba2124551310b6c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "606b7c6fac904807ba2124551310b6c8" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 40811 } } }
I20260620 23:02:16.175207 16804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 606b7c6fac904807ba2124551310b6c8 [sys.catalog]: This master's current role is: LEADER
I20260620 23:02:16.174620 16803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 606b7c6fac904807ba2124551310b6c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "606b7c6fac904807ba2124551310b6c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "606b7c6fac904807ba2124551310b6c8" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 40811 } } }
I20260620 23:02:16.175652 16803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 606b7c6fac904807ba2124551310b6c8 [sys.catalog]: This master's current role is: LEADER
I20260620 23:02:16.180250 16810 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:02:16.185245 16810 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:02:16.194061 16810 catalog_manager.cc:1347] Generated new cluster ID: 7bc5e61b806548cc80bb549f4061e1ad
I20260620 23:02:16.194202 16810 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:02:16.204054 16810 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:02:16.205451 16810 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:02:16.215276 16810 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 606b7c6fac904807ba2124551310b6c8: Generated new TSK 0
I20260620 23:02:16.216452 16810 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:02:16.272713 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 16730
I20260620 23:02:16.273979 16792 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:02:16.373883 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 16730
2026-06-20T23:02:16Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2062 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:02:16 dist-test-slave-f63b krb5kdc[16825](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:02:16 dist-test-slave-f63b krb5kdc[16825](info): set up 2 sockets
Jun 20 23:02:16 dist-test-slave-f63b krb5kdc[16825](info): commencing operation
krb5kdc: starting...
W20260620 23:02:17.479072 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.008s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 23:02:17 dist-test-slave-f63b krb5kdc[16825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:02:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:02:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
I20260620 23:02:17.726751 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:41897
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:39699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:41897
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:02:18.070071 16838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:02:18.070407 16838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:02:18.070662 16838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:02:18.078362 16838 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:02:18.078506 16838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:02:18.078686 16838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:02:18.078791 16838 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:02:18.078887 16838 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:02:18.089570 16838 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:39699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:41897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:41897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:02:18.091137 16838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:02:18.092602 16838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:02:18.106211 16844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:18.106256 16843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:18.106179 16846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:02:18.107065 16838 server_base.cc:1034] running on GCE node
I20260620 23:02:18.107918 16838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:02:18.109424 16838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:02:18.110720 16838 hybrid_clock.cc:648] HybridClock initialized: now 1781996538110744 us; error 124 us; skew 500 ppm
I20260620 23:02:18.111117 16838 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:02:18 dist-test-slave-f63b krb5kdc[16825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996538, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:02:18.118635 16838 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 23:02:18.120298 16838 webserver.cc:460] Webserver started at http://127.2.237.62:33987/ using document root <none> and password file <none>
I20260620 23:02:18.121055 16838 fs_manager.cc:359] Metadata directory not provided
I20260620 23:02:18.121215 16838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:02:18.121553 16838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:02:18.124778 16838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "ebcb4fd8961f46a3a3130026fcc631be"
format_stamp: "Formatted at 2026-06-20 23:02:18 on dist-test-slave-f63b"
server_key: "38f310ff8e6a3c37ed71cd09ff50bd9a"
server_key_iv: "9c55bd0b98672091dc24ee0e91588d91"
server_key_version: "encryptionkey@0"
I20260620 23:02:18.125576 16838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "ebcb4fd8961f46a3a3130026fcc631be"
format_stamp: "Formatted at 2026-06-20 23:02:18 on dist-test-slave-f63b"
server_key: "38f310ff8e6a3c37ed71cd09ff50bd9a"
server_key_iv: "9c55bd0b98672091dc24ee0e91588d91"
server_key_version: "encryptionkey@0"
I20260620 23:02:18.131125 16838 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260620 23:02:18.135092 16853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:18.136591 16838 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260620 23:02:18.136803 16838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "ebcb4fd8961f46a3a3130026fcc631be"
format_stamp: "Formatted at 2026-06-20 23:02:18 on dist-test-slave-f63b"
server_key: "38f310ff8e6a3c37ed71cd09ff50bd9a"
server_key_iv: "9c55bd0b98672091dc24ee0e91588d91"
server_key_version: "encryptionkey@0"
I20260620 23:02:18.137060 16838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:02:18.150388 16838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:02:18.154776 16838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:02:18.155086 16838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:02:18.174021 16838 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:41897
I20260620 23:02:18.174054 16904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:41897 every 8 connection(s)
I20260620 23:02:18.175505 16838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:02:18.179416 16905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:02:18.180270 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 16838
I20260620 23:02:18.180495 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:02:18.181372 2996 external_mini_cluster.cc:1428] Setting key 12d93ad5a440161dc75be723d57a97b0
I20260620 23:02:18.194388 16905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be: Bootstrap starting.
Jun 20 23:02:18 dist-test-slave-f63b krb5kdc[16825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:02:18.198160 16905 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be: Neither blocks nor log segments found. Creating new log.
I20260620 23:02:18.199350 16905 log.cc:824] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be: Log is configured to *not* fsync() on all Append() calls
I20260620 23:02:18.202755 16905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be: No bootstrap required, opened a new log
I20260620 23:02:18.206765 16908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:18.183639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34820 (local address 127.2.237.62:41897)
0620 23:02:18.184479 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:18.184491 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:18.184537 (+ 46us) server_negotiation.cc:407] Connection header received
0620 23:02:18.186244 (+ 1707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:18.186268 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:18.186765 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:18.187113 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:18.188855 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:18.190185 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:18.192152 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:18.192838 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:18.192973 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:18.197913 (+ 4940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:18.197920 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:18.197936 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:18.197973 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:18.201696 (+ 3723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:18.202526 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:18.202534 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:18.202542 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:18.202794 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:18.203230 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:18.203237 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:18.203243 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:18.203988 (+ 745us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:18.204304 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:18.204698 (+ 394us) server_negotiation.cc:299] Negotiation successful
0620 23:02:18.205194 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":409,"threads_started":1}
I20260620 23:02:18.211897 16905 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcb4fd8961f46a3a3130026fcc631be" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 41897 } }
I20260620 23:02:18.212280 16905 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:02:18.212363 16905 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebcb4fd8961f46a3a3130026fcc631be, State: Initialized, Role: FOLLOWER
I20260620 23:02:18.212852 16905 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcb4fd8961f46a3a3130026fcc631be" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 41897 } }
I20260620 23:02:18.213039 16905 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:02:18.213135 16905 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:02:18.213320 16905 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:02:18.215821 16905 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcb4fd8961f46a3a3130026fcc631be" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 41897 } }
I20260620 23:02:18.216246 16905 leader_election.cc:302] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebcb4fd8961f46a3a3130026fcc631be; no voters:
I20260620 23:02:18.216838 16905 leader_election.cc:288] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:02:18.216995 16910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:02:18.217643 16910 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [term 1 LEADER]: Becoming Leader. State: Replica: ebcb4fd8961f46a3a3130026fcc631be, State: Running, Role: LEADER
I20260620 23:02:18.218206 16910 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcb4fd8961f46a3a3130026fcc631be" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 41897 } }
I20260620 23:02:18.218961 16905 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:02:18.221068 16912 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebcb4fd8961f46a3a3130026fcc631be. Latest consensus state: current_term: 1 leader_uuid: "ebcb4fd8961f46a3a3130026fcc631be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcb4fd8961f46a3a3130026fcc631be" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 41897 } } }
I20260620 23:02:18.221498 16912 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [sys.catalog]: This master's current role is: LEADER
I20260620 23:02:18.222949 16911 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebcb4fd8961f46a3a3130026fcc631be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcb4fd8961f46a3a3130026fcc631be" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 41897 } } }
I20260620 23:02:18.223327 16911 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [sys.catalog]: This master's current role is: LEADER
I20260620 23:02:18.225198 16917 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:02:18.230793 16917 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:02:18.240353 16917 catalog_manager.cc:1347] Generated new cluster ID: 40332be05fe444e38fdd81d798e1fba8
I20260620 23:02:18.240525 16917 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:02:18.263005 16917 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:02:18.264415 16917 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:02:18.281006 16917 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be: Generated new TSK 0
I20260620 23:02:18.281960 16917 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.237.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
I20260620 23:02:18.354647 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:0
--local_ip_for_outbound_sockets=127.2.237.1
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:41897
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:39699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:02:18.694796 16931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:02:18.695153 16931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:02:18.695327 16931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:02:18.703145 16931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:02:18.703315 16931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:02:18.703557 16931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 23:02:18.715653 16931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:39699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:41897
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:02:18.717108 16931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:02:18.718469 16931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:02:18.732488 16936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:18.733057 16939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:18.733083 16937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:02:18.733758 16931 server_base.cc:1034] running on GCE node
I20260620 23:02:18.734494 16931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:02:18.735572 16931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:02:18.736822 16931 hybrid_clock.cc:648] HybridClock initialized: now 1781996538736755 us; error 60 us; skew 500 ppm
I20260620 23:02:18.737164 16931 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:02:18 dist-test-slave-f63b krb5kdc[16825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996538, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:02:18.740556 16931 init.cc:377] Logged in from keytab as kudu/127.2.237.1@KRBTEST.COM (short username kudu)
I20260620 23:02:18.742103 16931 webserver.cc:460] Webserver started at http://127.2.237.1:46483/ using document root <none> and password file <none>
I20260620 23:02:18.742954 16931 fs_manager.cc:359] Metadata directory not provided
I20260620 23:02:18.743110 16931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:02:18.743433 16931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:02:18.746500 16931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/data/instance:
uuid: "5e301f6cfd00456ebf43f313450dd98f"
format_stamp: "Formatted at 2026-06-20 23:02:18 on dist-test-slave-f63b"
server_key: "ecfcb230876b7c74e38815edb34821c6"
server_key_iv: "c2c8b6161538c8ad32e6fd89fde9925a"
server_key_version: "encryptionkey@0"
I20260620 23:02:18.747378 16931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance:
uuid: "5e301f6cfd00456ebf43f313450dd98f"
format_stamp: "Formatted at 2026-06-20 23:02:18 on dist-test-slave-f63b"
server_key: "ecfcb230876b7c74e38815edb34821c6"
server_key_iv: "c2c8b6161538c8ad32e6fd89fde9925a"
server_key_version: "encryptionkey@0"
I20260620 23:02:18.752949 16931 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260620 23:02:18.756647 16946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:18.758196 16931 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 23:02:18.758402 16931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "5e301f6cfd00456ebf43f313450dd98f"
format_stamp: "Formatted at 2026-06-20 23:02:18 on dist-test-slave-f63b"
server_key: "ecfcb230876b7c74e38815edb34821c6"
server_key_iv: "c2c8b6161538c8ad32e6fd89fde9925a"
server_key_version: "encryptionkey@0"
I20260620 23:02:18.758692 16931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:02:18.772192 16931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:02:18.776582 16931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:02:18.776904 16931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:02:18.777935 16931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:02:18.779469 16931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:02:18.779578 16931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:18.779745 16931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:02:18.779826 16931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:18.814815 16931 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:39915
I20260620 23:02:18.814858 17058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:39915 every 8 connection(s)
I20260620 23:02:18.816248 16931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 23:02:18.821704 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 16931
I20260620 23:02:18.821957 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance
I20260620 23:02:18.822813 2996 external_mini_cluster.cc:1428] Setting key c6d6981aad41565ec9a23fc799620bec
Jun 20 23:02:18 dist-test-slave-f63b krb5kdc[16825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996538, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:02:18.840667 17061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:18.819091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:57687 (local address 127.2.237.62:41897)
0620 23:02:18.819603 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:18.819609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:18.820883 (+ 1274us) server_negotiation.cc:407] Connection header received
0620 23:02:18.823712 (+ 2829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:18.823717 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:18.823828 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:18.823955 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:18.825862 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:18.826630 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:18.828277 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:18.828670 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:18.828820 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:18.832987 (+ 4167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:18.832996 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:18.833004 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:18.833055 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:18.836750 (+ 3695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:18.837637 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:18.837646 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:18.837652 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:18.837817 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:18.838340 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:18.838348 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:18.838354 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:18.838829 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:18.839000 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:18.839832 (+ 832us) server_negotiation.cc:299] Negotiation successful
0620 23:02:18.840148 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":251,"threads_started":1}
I20260620 23:02:18.841539 17062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:18.819843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.1:57687)
0620 23:02:18.820701 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:18.820742 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:18.823355 (+ 2613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:18.824215 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:18.824224 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:18.824881 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:02:18.825657 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:18.825680 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:18.826798 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:18.826804 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:18.828103 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:18.828115 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:18.828905 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:18.828914 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:18.829169 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:18.829952 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:02:18.829975 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:02:18.832684 (+ 2709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:02:18.836944 (+ 4260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:18.836954 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:18.836971 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:18.837493 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:18.837948 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:18.837951 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:18.837954 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:18.838200 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:18.838999 (+ 799us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:02:18.839006 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:02:18.839415 (+ 409us) client_negotiation.cc:769] Sending connection context
0620 23:02:18.839681 (+ 266us) client_negotiation.cc:240] Negotiation successful
0620 23:02:18.840220 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":347,"threads_started":1}
I20260620 23:02:18.843401 17059 heartbeater.cc:347] Connected to a master server at 127.2.237.62:41897
I20260620 23:02:18.843827 17059 heartbeater.cc:464] Registering TS with master...
I20260620 23:02:18.844689 17059 heartbeater.cc:511] Master 127.2.237.62:41897 requested a full tablet report, sending...
I20260620 23:02:18.847152 16870 ts_manager.cc:194] Registered new tserver with Master: 5e301f6cfd00456ebf43f313450dd98f (127.2.237.1:39915)
WARNING: no policy specified for kudu/127.2.237.2@KRBTEST.COM; defaulting to no policy
I20260620 23:02:18.848850 16870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.1@KRBTEST.COM'} at 127.2.237.1:57687
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
I20260620 23:02:18.878830 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:0
--local_ip_for_outbound_sockets=127.2.237.2
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:41897
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:39699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:02:19.222942 17065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:02:19.223279 17065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:02:19.223379 17065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:02:19.231101 17065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:02:19.231225 17065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:02:19.231346 17065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 23:02:19.241528 17065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:39699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:41897
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:02:19.243032 17065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:02:19.244419 17065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:02:19.258248 17071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:19.258193 17073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:19.258285 17070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:02:19.259402 17065 server_base.cc:1034] running on GCE node
I20260620 23:02:19.259985 17065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:02:19.260910 17065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:02:19.262113 17065 hybrid_clock.cc:648] HybridClock initialized: now 1781996539262049 us; error 47 us; skew 500 ppm
I20260620 23:02:19.262663 17065 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:02:19 dist-test-slave-f63b krb5kdc[16825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996539, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:02:19.266368 17065 init.cc:377] Logged in from keytab as kudu/127.2.237.2@KRBTEST.COM (short username kudu)
I20260620 23:02:19.268031 17065 webserver.cc:460] Webserver started at http://127.2.237.2:44703/ using document root <none> and password file <none>
I20260620 23:02:19.268762 17065 fs_manager.cc:359] Metadata directory not provided
I20260620 23:02:19.268915 17065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:02:19.269241 17065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:02:19.272521 17065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/data/instance:
uuid: "4dc7a8b876e149bd9f773b7cac8a0c30"
format_stamp: "Formatted at 2026-06-20 23:02:19 on dist-test-slave-f63b"
server_key: "6f5759a55668bbbde0588f820884d01b"
server_key_iv: "86f08585b37800801857e2f3debf34d5"
server_key_version: "encryptionkey@0"
I20260620 23:02:19.273350 17065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance:
uuid: "4dc7a8b876e149bd9f773b7cac8a0c30"
format_stamp: "Formatted at 2026-06-20 23:02:19 on dist-test-slave-f63b"
server_key: "6f5759a55668bbbde0588f820884d01b"
server_key_iv: "86f08585b37800801857e2f3debf34d5"
server_key_version: "encryptionkey@0"
I20260620 23:02:19.279120 17065 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260620 23:02:19.283443 17080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:19.285148 17065 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 23:02:19.285375 17065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "4dc7a8b876e149bd9f773b7cac8a0c30"
format_stamp: "Formatted at 2026-06-20 23:02:19 on dist-test-slave-f63b"
server_key: "6f5759a55668bbbde0588f820884d01b"
server_key_iv: "86f08585b37800801857e2f3debf34d5"
server_key_version: "encryptionkey@0"
I20260620 23:02:19.285616 17065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:02:19.326992 17065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:02:19.331703 17065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:02:19.332077 17065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:02:19.333195 17065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:02:19.334637 17065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:02:19.334721 17065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260620 23:02:19.334811 17065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:02:19.334863 17065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:19.369782 17065 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:46845
I20260620 23:02:19.369835 17192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:46845 every 8 connection(s)
I20260620 23:02:19.371197 17065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 23:02:19.373495 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 17065
I20260620 23:02:19.373718 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance
I20260620 23:02:19.374310 2996 external_mini_cluster.cc:1428] Setting key 457d738f7c429197ca72a5a822aefa31
Jun 20 23:02:19 dist-test-slave-f63b krb5kdc[16825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996539, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:02:19.393973 17195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:19.373892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:38751 (local address 127.2.237.62:41897)
0620 23:02:19.374374 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:19.374381 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:19.375561 (+ 1180us) server_negotiation.cc:407] Connection header received
0620 23:02:19.377397 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:19.377406 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:19.377501 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:19.377623 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:19.379228 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:19.380435 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:19.382318 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:19.382855 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:19.383032 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:19.387145 (+ 4113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:19.387151 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:19.387154 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:19.387193 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:19.390224 (+ 3031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:19.391235 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:19.391245 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:19.391248 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:19.391394 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:19.391909 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:19.391914 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:19.391916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:19.392240 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:19.392423 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:19.393255 (+ 832us) server_negotiation.cc:299] Negotiation successful
0620 23:02:19.393512 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":228,"threads_started":1}
I20260620 23:02:19.395164 17196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:19.374479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.2:38751)
0620 23:02:19.375389 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:19.375444 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:19.377130 (+ 1686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:19.377851 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:19.377859 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:19.378364 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:02:19.379040 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:19.379060 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:19.380629 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:19.380634 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:19.382126 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:19.382141 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:19.383113 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:19.383123 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:19.383407 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:19.384164 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:02:19.384190 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:02:19.386863 (+ 2673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:02:19.390393 (+ 3530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:19.390403 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:19.390429 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:19.391078 (+ 649us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:19.391531 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:19.391537 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:19.391541 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:19.391766 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:19.392395 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:02:19.392405 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:02:19.392824 (+ 419us) client_negotiation.cc:769] Sending connection context
0620 23:02:19.393134 (+ 310us) client_negotiation.cc:240] Negotiation successful
0620 23:02:19.393749 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":336,"threads_started":1}
I20260620 23:02:19.396929 17193 heartbeater.cc:347] Connected to a master server at 127.2.237.62:41897
WARNING: no policy specified for kudu/127.2.237.3@KRBTEST.COM; defaulting to no policy
I20260620 23:02:19.397331 17193 heartbeater.cc:464] Registering TS with master...
I20260620 23:02:19.398161 17193 heartbeater.cc:511] Master 127.2.237.62:41897 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.3@KRBTEST.COM" created.
I20260620 23:02:19.399941 16870 ts_manager.cc:194] Registered new tserver with Master: 4dc7a8b876e149bd9f773b7cac8a0c30 (127.2.237.2:46845)
I20260620 23:02:19.400945 16870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.2@KRBTEST.COM'} at 127.2.237.2:38751
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
I20260620 23:02:19.428349 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:0
--local_ip_for_outbound_sockets=127.2.237.3
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:41897
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:39699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:02:19.764211 17199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:02:19.764503 17199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:02:19.764654 17199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:02:19.772456 17199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:02:19.772580 17199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:02:19.772769 17199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 23:02:19.782922 17199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:39699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:41897
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:02:19.784300 17199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:02:19.785662 17199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:02:19.799291 17205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:19.799248 17204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:19.799326 17207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:02:19.800196 17199 server_base.cc:1034] running on GCE node
I20260620 23:02:19.800916 17199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:02:19.802019 17199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:02:19.803251 17199 hybrid_clock.cc:648] HybridClock initialized: now 1781996539803204 us; error 31 us; skew 500 ppm
I20260620 23:02:19.803580 17199 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:02:19 dist-test-slave-f63b krb5kdc[16825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996539, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:02:19.806890 17199 init.cc:377] Logged in from keytab as kudu/127.2.237.3@KRBTEST.COM (short username kudu)
I20260620 23:02:19.808508 17199 webserver.cc:460] Webserver started at http://127.2.237.3:36581/ using document root <none> and password file <none>
I20260620 23:02:19.809192 17199 fs_manager.cc:359] Metadata directory not provided
I20260620 23:02:19.809298 17199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:02:19.809557 17199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:02:19.812606 17199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/data/instance:
uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46"
format_stamp: "Formatted at 2026-06-20 23:02:19 on dist-test-slave-f63b"
server_key: "3c4f4f339d2cfd06e390ac65416f523f"
server_key_iv: "fc7c874cb24c79f50893da20c674d697"
server_key_version: "encryptionkey@0"
I20260620 23:02:19.813336 17199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance:
uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46"
format_stamp: "Formatted at 2026-06-20 23:02:19 on dist-test-slave-f63b"
server_key: "3c4f4f339d2cfd06e390ac65416f523f"
server_key_iv: "fc7c874cb24c79f50893da20c674d697"
server_key_version: "encryptionkey@0"
I20260620 23:02:19.818693 17199 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260620 23:02:19.822333 17214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:19.823772 17199 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260620 23:02:19.823906 17199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46"
format_stamp: "Formatted at 2026-06-20 23:02:19 on dist-test-slave-f63b"
server_key: "3c4f4f339d2cfd06e390ac65416f523f"
server_key_iv: "fc7c874cb24c79f50893da20c674d697"
server_key_version: "encryptionkey@0"
I20260620 23:02:19.824080 17199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:02:19.847637 17199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:02:19.852393 17199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:02:19.852397 17059 heartbeater.cc:503] Master 127.2.237.62:41897 was elected leader, sending a full tablet report...
I20260620 23:02:19.852720 17199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:02:19.853969 17199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:02:19.855587 17199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:02:19.855664 17199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:19.855751 17199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:02:19.855798 17199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:19.891129 17199 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:44217
I20260620 23:02:19.891185 17326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:44217 every 8 connection(s)
I20260620 23:02:19.892536 17199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 23:02:19.902488 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 17199
I20260620 23:02:19.902711 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance
I20260620 23:02:19.903298 2996 external_mini_cluster.cc:1428] Setting key 16656519b706d72cc9ba864f6b457815
Jun 20 23:02:19 dist-test-slave-f63b krb5kdc[16825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996539, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:02:19.914716 17329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:19.895263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:52021 (local address 127.2.237.62:41897)
0620 23:02:19.895740 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:19.895745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:19.896847 (+ 1102us) server_negotiation.cc:407] Connection header received
0620 23:02:19.898780 (+ 1933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:19.898785 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:19.898855 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:19.898963 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:19.900440 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:19.901234 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:19.903384 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:19.903835 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:19.903961 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:19.908349 (+ 4388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:19.908354 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:19.908357 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:19.908393 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:19.911468 (+ 3075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:19.912247 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:19.912252 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:19.912256 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:19.912387 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:19.912849 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:19.912858 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:19.912860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:19.913144 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:19.913314 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:19.914078 (+ 764us) server_negotiation.cc:299] Negotiation successful
0620 23:02:19.914305 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":240,"threads_started":1}
I20260620 23:02:19.915757 17330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:19.895829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.3:52021)
0620 23:02:19.896668 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:19.896724 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:19.898461 (+ 1737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:19.899171 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:19.899179 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:19.899696 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:02:19.900276 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:19.900295 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:19.901370 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:19.901373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:19.903211 (+ 1838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:19.903227 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:19.904091 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:19.904098 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:19.904354 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:19.905215 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:02:19.905247 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:02:19.908096 (+ 2849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:02:19.911623 (+ 3527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:19.911632 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:19.911651 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:19.912106 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:19.912506 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:19.912511 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:19.912515 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:19.912721 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:19.913293 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:02:19.913301 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:02:19.913686 (+ 385us) client_negotiation.cc:769] Sending connection context
0620 23:02:19.913966 (+ 280us) client_negotiation.cc:240] Negotiation successful
0620 23:02:19.914553 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":356,"threads_started":1}
I20260620 23:02:19.917277 17327 heartbeater.cc:347] Connected to a master server at 127.2.237.62:41897
I20260620 23:02:19.917639 17327 heartbeater.cc:464] Registering TS with master...
I20260620 23:02:19.918545 17327 heartbeater.cc:511] Master 127.2.237.62:41897 requested a full tablet report, sending...
I20260620 23:02:19.920136 16870 ts_manager.cc:194] Registered new tserver with Master: 5f09a35ebd6f4bb0bbe5799d06ac3a46 (127.2.237.3:44217)
I20260620 23:02:19.920965 16870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.3@KRBTEST.COM'} at 127.2.237.3:52021
I20260620 23:02:19.931926 2996 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:02:19.950819 17329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:19.936607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34836 (local address 127.2.237.62:41897)
0620 23:02:19.936854 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:19.936862 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:19.937248 (+ 386us) server_negotiation.cc:407] Connection header received
0620 23:02:19.937465 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:19.937471 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:19.937548 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:19.937670 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:19.939147 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:19.939908 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:19.941497 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:19.941941 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:19.942090 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:19.944839 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:19.944845 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:19.944848 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:19.944883 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:19.947622 (+ 2739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:19.948442 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:19.948448 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:19.948451 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:19.948606 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:19.949059 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:19.949066 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:19.949069 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:19.949464 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:19.949772 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:19.950165 (+ 393us) server_negotiation.cc:299] Negotiation successful
0620 23:02:19.950425 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260620 23:02:19.955277 16870 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34836:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:02:19.962257 16870 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:02:20.005263 17342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:19.976283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50678 (local address 127.2.237.1:39915)
0620 23:02:19.997119 (+ 20836us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:19.997127 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:19.997153 (+ 26us) server_negotiation.cc:407] Connection header received
0620 23:02:19.997251 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:19.997258 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:19.997528 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:19.998456 (+ 928us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:02:19.998769 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:20.000187 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:20.002652 (+ 2465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:20.003981 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:20.004139 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:20.004409 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:20.004676 (+ 267us) server_negotiation.cc:299] Negotiation successful
0620 23:02:20.004855 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20664,"thread_start_us":20510,"threads_started":1}
I20260620 23:02:20.005285 17341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:19.976078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:39915 (local address 127.0.0.1:50678)
0620 23:02:19.988925 (+ 12847us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:19.988990 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:19.989278 (+ 288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:19.997813 (+ 8535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:19.997819 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:19.997843 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:02:19.998261 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:19.998271 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:20.000326 (+ 2055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:20.000335 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:20.002429 (+ 2094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:20.002445 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:20.004198 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:20.004204 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:20.004407 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:20.004469 (+ 62us) client_negotiation.cc:769] Sending connection context
0620 23:02:20.004631 (+ 162us) client_negotiation.cc:240] Negotiation successful
0620 23:02:20.004860 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12433,"mutex_wait_us":128,"thread_start_us":15646,"threads_started":1}
I20260620 23:02:20.006763 17343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:19.976414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47678 (local address 127.2.237.2:46845)
0620 23:02:19.996574 (+ 20160us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:19.996582 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:19.996608 (+ 26us) server_negotiation.cc:407] Connection header received
0620 23:02:19.996693 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:19.996700 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:19.996949 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:19.997841 (+ 892us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:02:19.998087 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:19.999673 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:20.003396 (+ 3723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:20.005030 (+ 1634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:20.005173 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:20.005392 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:20.006258 (+ 866us) server_negotiation.cc:299] Negotiation successful
0620 23:02:20.006425 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20003,"thread_start_us":19915,"threads_started":1}
I20260620 23:02:20.006894 17345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:19.979248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52124 (local address 127.2.237.3:44217)
0620 23:02:19.996299 (+ 17051us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:19.996311 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:19.996355 (+ 44us) server_negotiation.cc:407] Connection header received
0620 23:02:19.996478 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:19.996487 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:19.996787 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:19.997020 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:02:19.998071 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:19.999798 (+ 1727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:20.003397 (+ 3599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:20.005030 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:20.005173 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:20.005425 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:20.006311 (+ 886us) server_negotiation.cc:299] Negotiation successful
0620 23:02:20.006497 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16831,"thread_start_us":16656,"threads_started":1}
I20260620 23:02:20.007371 17340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:19.976170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:46845 (local address 127.0.0.1:47678)
0620 23:02:19.991420 (+ 15250us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:19.991439 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:19.991565 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:19.997187 (+ 5622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:19.997192 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:19.997222 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:02:19.997778 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:19.997797 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:19.999836 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:19.999846 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:20.003181 (+ 3335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:20.003193 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:20.005597 (+ 2404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:20.005604 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:20.005891 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:20.006074 (+ 183us) client_negotiation.cc:769] Sending connection context
0620 23:02:20.006176 (+ 102us) client_negotiation.cc:240] Negotiation successful
0620 23:02:20.006360 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15122,"mutex_wait_us":422,"thread_start_us":13245,"threads_started":1}
I20260620 23:02:20.007450 17344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:19.978524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:44217 (local address 127.0.0.1:52124)
0620 23:02:19.995394 (+ 16870us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:19.995422 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:19.995588 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:19.997121 (+ 1533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:19.997129 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:19.997164 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:02:19.997831 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:19.997843 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:19.999945 (+ 2102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:19.999951 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:20.003176 (+ 3225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:20.003193 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:20.005611 (+ 2418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:20.005617 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:20.005890 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:20.006074 (+ 184us) client_negotiation.cc:769] Sending connection context
0620 23:02:20.006174 (+ 100us) client_negotiation.cc:240] Negotiation successful
0620 23:02:20.006959 (+ 785us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16664,"mutex_wait_us":845,"thread_start_us":16952,"threads_started":1}
I20260620 23:02:20.010144 16994 tablet_service.cc:1467] Processing CreateTablet for tablet bc57f60850374f6396600477a0d37cac (DEFAULT_TABLE table=test-table [id=57ca0eea5f884a85a2ef80070776a94c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:02:20.011675 16994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc57f60850374f6396600477a0d37cac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:02:20.011186 17128 tablet_service.cc:1467] Processing CreateTablet for tablet bc57f60850374f6396600477a0d37cac (DEFAULT_TABLE table=test-table [id=57ca0eea5f884a85a2ef80070776a94c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:02:20.011791 17262 tablet_service.cc:1467] Processing CreateTablet for tablet bc57f60850374f6396600477a0d37cac (DEFAULT_TABLE table=test-table [id=57ca0eea5f884a85a2ef80070776a94c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:02:20.012625 17128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc57f60850374f6396600477a0d37cac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:02:20.013135 17262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc57f60850374f6396600477a0d37cac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:02:20.025374 17346 tablet_bootstrap.cc:492] T bc57f60850374f6396600477a0d37cac P 5e301f6cfd00456ebf43f313450dd98f: Bootstrap starting.
I20260620 23:02:20.029196 17346 tablet_bootstrap.cc:654] T bc57f60850374f6396600477a0d37cac P 5e301f6cfd00456ebf43f313450dd98f: Neither blocks nor log segments found. Creating new log.
I20260620 23:02:20.030649 17346 log.cc:824] T bc57f60850374f6396600477a0d37cac P 5e301f6cfd00456ebf43f313450dd98f: Log is configured to *not* fsync() on all Append() calls
I20260620 23:02:20.032411 17347 tablet_bootstrap.cc:492] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30: Bootstrap starting.
I20260620 23:02:20.032279 17348 tablet_bootstrap.cc:492] T bc57f60850374f6396600477a0d37cac P 5f09a35ebd6f4bb0bbe5799d06ac3a46: Bootstrap starting.
I20260620 23:02:20.036140 17347 tablet_bootstrap.cc:654] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30: Neither blocks nor log segments found. Creating new log.
I20260620 23:02:20.036998 17348 tablet_bootstrap.cc:654] T bc57f60850374f6396600477a0d37cac P 5f09a35ebd6f4bb0bbe5799d06ac3a46: Neither blocks nor log segments found. Creating new log.
I20260620 23:02:20.037358 17347 log.cc:824] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30: Log is configured to *not* fsync() on all Append() calls
I20260620 23:02:20.038442 17348 log.cc:824] T bc57f60850374f6396600477a0d37cac P 5f09a35ebd6f4bb0bbe5799d06ac3a46: Log is configured to *not* fsync() on all Append() calls
I20260620 23:02:20.040920 17346 tablet_bootstrap.cc:492] T bc57f60850374f6396600477a0d37cac P 5e301f6cfd00456ebf43f313450dd98f: No bootstrap required, opened a new log
I20260620 23:02:20.041252 17346 ts_tablet_manager.cc:1397] T bc57f60850374f6396600477a0d37cac P 5e301f6cfd00456ebf43f313450dd98f: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.009s
I20260620 23:02:20.041460 17347 tablet_bootstrap.cc:492] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30: No bootstrap required, opened a new log
I20260620 23:02:20.041728 17347 ts_tablet_manager.cc:1397] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260620 23:02:20.042461 17348 tablet_bootstrap.cc:492] T bc57f60850374f6396600477a0d37cac P 5f09a35ebd6f4bb0bbe5799d06ac3a46: No bootstrap required, opened a new log
I20260620 23:02:20.042788 17348 ts_tablet_manager.cc:1397] T bc57f60850374f6396600477a0d37cac P 5f09a35ebd6f4bb0bbe5799d06ac3a46: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260620 23:02:20.051366 17348 raft_consensus.cc:348] T bc57f60850374f6396600477a0d37cac P 5f09a35ebd6f4bb0bbe5799d06ac3a46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e301f6cfd00456ebf43f313450dd98f" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 39915 } } peers { permanent_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46845 } } peers { permanent_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44217 } }
I20260620 23:02:20.051846 17348 raft_consensus.cc:374] T bc57f60850374f6396600477a0d37cac P 5f09a35ebd6f4bb0bbe5799d06ac3a46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:02:20.051987 17348 raft_consensus.cc:729] T bc57f60850374f6396600477a0d37cac P 5f09a35ebd6f4bb0bbe5799d06ac3a46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f09a35ebd6f4bb0bbe5799d06ac3a46, State: Initialized, Role: FOLLOWER
I20260620 23:02:20.052753 17348 consensus_queue.cc:260] T bc57f60850374f6396600477a0d37cac P 5f09a35ebd6f4bb0bbe5799d06ac3a46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e301f6cfd00456ebf43f313450dd98f" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 39915 } } peers { permanent_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46845 } } peers { permanent_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44217 } }
I20260620 23:02:20.052608 17346 raft_consensus.cc:348] T bc57f60850374f6396600477a0d37cac P 5e301f6cfd00456ebf43f313450dd98f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e301f6cfd00456ebf43f313450dd98f" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 39915 } } peers { permanent_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46845 } } peers { permanent_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44217 } }
I20260620 23:02:20.052608 17347 raft_consensus.cc:348] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e301f6cfd00456ebf43f313450dd98f" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 39915 } } peers { permanent_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46845 } } peers { permanent_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44217 } }
I20260620 23:02:20.053068 17346 raft_consensus.cc:374] T bc57f60850374f6396600477a0d37cac P 5e301f6cfd00456ebf43f313450dd98f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:02:20.053063 17347 raft_consensus.cc:374] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:02:20.053172 17347 raft_consensus.cc:729] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dc7a8b876e149bd9f773b7cac8a0c30, State: Initialized, Role: FOLLOWER
I20260620 23:02:20.053223 17346 raft_consensus.cc:729] T bc57f60850374f6396600477a0d37cac P 5e301f6cfd00456ebf43f313450dd98f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e301f6cfd00456ebf43f313450dd98f, State: Initialized, Role: FOLLOWER
I20260620 23:02:20.054028 17327 heartbeater.cc:503] Master 127.2.237.62:41897 was elected leader, sending a full tablet report...
I20260620 23:02:20.054024 17346 consensus_queue.cc:260] T bc57f60850374f6396600477a0d37cac P 5e301f6cfd00456ebf43f313450dd98f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e301f6cfd00456ebf43f313450dd98f" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 39915 } } peers { permanent_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46845 } } peers { permanent_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44217 } }
I20260620 23:02:20.054311 17347 consensus_queue.cc:260] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e301f6cfd00456ebf43f313450dd98f" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 39915 } } peers { permanent_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46845 } } peers { permanent_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44217 } }
I20260620 23:02:20.055449 17348 ts_tablet_manager.cc:1428] T bc57f60850374f6396600477a0d37cac P 5f09a35ebd6f4bb0bbe5799d06ac3a46: Time spent starting tablet: real 0.013s user 0.007s sys 0.004s
I20260620 23:02:20.057277 17346 ts_tablet_manager.cc:1428] T bc57f60850374f6396600477a0d37cac P 5e301f6cfd00456ebf43f313450dd98f: Time spent starting tablet: real 0.016s user 0.016s sys 0.001s
I20260620 23:02:20.058808 17193 heartbeater.cc:503] Master 127.2.237.62:41897 was elected leader, sending a full tablet report...
I20260620 23:02:20.059479 17347 ts_tablet_manager.cc:1428] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30: Time spent starting tablet: real 0.018s user 0.015s sys 0.000s
W20260620 23:02:20.068599 17060 tablet.cc:2369] T bc57f60850374f6396600477a0d37cac P 5e301f6cfd00456ebf43f313450dd98f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:02:20.123386 17194 tablet.cc:2369] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:02:20.144178 17328 tablet.cc:2369] T bc57f60850374f6396600477a0d37cac P 5f09a35ebd6f4bb0bbe5799d06ac3a46: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:02:20.160929 17354 raft_consensus.cc:482] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:02:20.161329 17354 raft_consensus.cc:504] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e301f6cfd00456ebf43f313450dd98f" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 39915 } } peers { permanent_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46845 } } peers { permanent_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44217 } }
I20260620 23:02:20.163161 17354 leader_election.cc:288] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e301f6cfd00456ebf43f313450dd98f (127.2.237.1:39915), 5f09a35ebd6f4bb0bbe5799d06ac3a46 (127.2.237.3:44217)
I20260620 23:02:20.169950 17342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:20.164198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:56875 (local address 127.2.237.1:39915)
0620 23:02:20.164408 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:20.164414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:20.164434 (+ 20us) server_negotiation.cc:407] Connection header received
0620 23:02:20.164662 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:20.164666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:20.164728 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:20.164837 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:02:20.165369 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:20.166112 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:20.168091 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:20.168878 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:20.169055 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:20.169203 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:20.169533 (+ 330us) server_negotiation.cc:299] Negotiation successful
0620 23:02:20.169666 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260620 23:02:20.170087 17345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:20.163787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:54433 (local address 127.2.237.3:44217)
0620 23:02:20.164038 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:20.164044 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:20.164795 (+ 751us) server_negotiation.cc:407] Connection header received
0620 23:02:20.164993 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:20.165002 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:20.165063 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:20.165174 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:02:20.165754 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:20.166468 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:20.168291 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:20.169109 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:20.169302 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:20.169411 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:20.169693 (+ 282us) server_negotiation.cc:299] Negotiation successful
0620 23:02:20.169825 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260620 23:02:20.170162 17355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:20.163541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:39915 (local address 127.2.237.2:56875)
0620 23:02:20.164280 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:20.164302 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:20.164497 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:20.164846 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:20.164851 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:20.164877 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:02:20.165203 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:20.165212 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:20.166245 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:20.166253 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:20.167925 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:20.167937 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:20.169150 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:20.169156 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:20.169359 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:20.169385 (+ 26us) client_negotiation.cc:769] Sending connection context
0620 23:02:20.169506 (+ 121us) client_negotiation.cc:240] Negotiation successful
0620 23:02:20.169649 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"mutex_wait_us":43,"thread_start_us":386,"threads_started":1}
I20260620 23:02:20.170259 17356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:20.163651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:44217 (local address 127.2.237.2:54433)
0620 23:02:20.164650 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:20.164683 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:20.164842 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:20.165202 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:20.165206 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:20.165245 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:02:20.165579 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:20.165587 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:20.166645 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:20.166651 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:20.168131 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:20.168143 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:20.169355 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:20.169361 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:20.169534 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:20.169558 (+ 24us) client_negotiation.cc:769] Sending connection context
0620 23:02:20.169674 (+ 116us) client_negotiation.cc:240] Negotiation successful
0620 23:02:20.169806 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":902,"mutex_wait_us":40,"thread_start_us":663,"threads_started":1}
I20260620 23:02:20.171324 17014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc57f60850374f6396600477a0d37cac" candidate_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e301f6cfd00456ebf43f313450dd98f" is_pre_election: true
I20260620 23:02:20.171363 17282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc57f60850374f6396600477a0d37cac" candidate_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46" is_pre_election: true
I20260620 23:02:20.171847 17282 raft_consensus.cc:2454] T bc57f60850374f6396600477a0d37cac P 5f09a35ebd6f4bb0bbe5799d06ac3a46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dc7a8b876e149bd9f773b7cac8a0c30 in term 0.
I20260620 23:02:20.171857 17014 raft_consensus.cc:2454] T bc57f60850374f6396600477a0d37cac P 5e301f6cfd00456ebf43f313450dd98f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dc7a8b876e149bd9f773b7cac8a0c30 in term 0.
I20260620 23:02:20.172554 17081 leader_election.cc:302] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4dc7a8b876e149bd9f773b7cac8a0c30, 5e301f6cfd00456ebf43f313450dd98f, 5f09a35ebd6f4bb0bbe5799d06ac3a46; no voters:
I20260620 23:02:20.172968 17354 raft_consensus.cc:2781] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:02:20.173123 17354 raft_consensus.cc:482] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:02:20.173292 17354 raft_consensus.cc:3037] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:02:20.176523 17354 raft_consensus.cc:504] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e301f6cfd00456ebf43f313450dd98f" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 39915 } } peers { permanent_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46845 } } peers { permanent_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44217 } }
I20260620 23:02:20.177220 17354 leader_election.cc:288] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30 [CANDIDATE]: Term 1 election: Requested vote from peers 5e301f6cfd00456ebf43f313450dd98f (127.2.237.1:39915), 5f09a35ebd6f4bb0bbe5799d06ac3a46 (127.2.237.3:44217)
I20260620 23:02:20.177701 17014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc57f60850374f6396600477a0d37cac" candidate_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e301f6cfd00456ebf43f313450dd98f"
I20260620 23:02:20.177748 17282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc57f60850374f6396600477a0d37cac" candidate_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46"
I20260620 23:02:20.177930 17014 raft_consensus.cc:3037] T bc57f60850374f6396600477a0d37cac P 5e301f6cfd00456ebf43f313450dd98f [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:02:20.177965 17282 raft_consensus.cc:3037] T bc57f60850374f6396600477a0d37cac P 5f09a35ebd6f4bb0bbe5799d06ac3a46 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:02:20.180238 17014 raft_consensus.cc:2454] T bc57f60850374f6396600477a0d37cac P 5e301f6cfd00456ebf43f313450dd98f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dc7a8b876e149bd9f773b7cac8a0c30 in term 1.
I20260620 23:02:20.180707 17081 leader_election.cc:302] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dc7a8b876e149bd9f773b7cac8a0c30, 5e301f6cfd00456ebf43f313450dd98f; no voters:
I20260620 23:02:20.180992 17354 raft_consensus.cc:2781] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:02:20.181118 17282 raft_consensus.cc:2454] T bc57f60850374f6396600477a0d37cac P 5f09a35ebd6f4bb0bbe5799d06ac3a46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dc7a8b876e149bd9f773b7cac8a0c30 in term 1.
I20260620 23:02:20.181556 17354 raft_consensus.cc:686] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30 [term 1 LEADER]: Becoming Leader. State: Replica: 4dc7a8b876e149bd9f773b7cac8a0c30, State: Running, Role: LEADER
I20260620 23:02:20.182142 17354 consensus_queue.cc:237] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e301f6cfd00456ebf43f313450dd98f" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 39915 } } peers { permanent_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46845 } } peers { permanent_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44217 } }
I20260620 23:02:20.187353 16870 catalog_manager.cc:5495] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dc7a8b876e149bd9f773b7cac8a0c30 (127.2.237.2). New cstate: current_term: 1 leader_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e301f6cfd00456ebf43f313450dd98f" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 39915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44217 } health_report { overall_health: UNKNOWN } } }
I20260620 23:02:20.244176 17343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:20.238690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47694 (local address 127.2.237.2:46845)
0620 23:02:20.238927 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:20.238933 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:20.238953 (+ 20us) server_negotiation.cc:407] Connection header received
0620 23:02:20.239130 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:20.239134 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:20.239224 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:20.239393 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:02:20.239770 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:20.240488 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:20.241926 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:20.242331 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:20.242466 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:20.243024 (+ 558us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:02:20.243570 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:02:20.243736 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:20.243795 (+ 59us) server_negotiation.cc:299] Negotiation successful
0620 23:02:20.243894 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260620 23:02:20.253185 17282 raft_consensus.cc:1261] T bc57f60850374f6396600477a0d37cac P 5f09a35ebd6f4bb0bbe5799d06ac3a46 [term 1 FOLLOWER]: Refusing update from remote peer 4dc7a8b876e149bd9f773b7cac8a0c30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:02:20.253185 17014 raft_consensus.cc:1261] T bc57f60850374f6396600477a0d37cac P 5e301f6cfd00456ebf43f313450dd98f [term 1 FOLLOWER]: Refusing update from remote peer 4dc7a8b876e149bd9f773b7cac8a0c30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:02:20.254256 17357 consensus_queue.cc:1035] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e301f6cfd00456ebf43f313450dd98f" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 39915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:02:20.254649 17354 consensus_queue.cc:1035] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:02:20.267418 17361 mvcc.cc:205] Tried to move back new op lower bound from 7299057828864958464 to 7299057828590276608. Current Snapshot: MvccSnapshot[applied={T|T < 7299057828864958464}]
I20260620 23:02:20.284442 16870 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:02:20.284778 16870 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34836:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:02:20.288916 16870 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be: Sending DeleteTablet for 3 replicas of tablet bc57f60850374f6396600477a0d37cac
I20260620 23:02:20.290233 17262 tablet_service.cc:1514] Processing DeleteTablet for tablet bc57f60850374f6396600477a0d37cac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:02:20 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:52124
I20260620 23:02:20.290232 17128 tablet_service.cc:1514] Processing DeleteTablet for tablet bc57f60850374f6396600477a0d37cac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:02:20 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:47678
I20260620 23:02:20.290277 16994 tablet_service.cc:1514] Processing DeleteTablet for tablet bc57f60850374f6396600477a0d37cac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:02:20 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:50678
I20260620 23:02:20.296087 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 16931
I20260620 23:02:20.303661 17371 tablet_replica.cc:330] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30: stopping tablet replica
I20260620 23:02:20.304003 17372 tablet_replica.cc:330] T bc57f60850374f6396600477a0d37cac P 5f09a35ebd6f4bb0bbe5799d06ac3a46: stopping tablet replica
I20260620 23:02:20.304244 17371 raft_consensus.cc:2229] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30 [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:02:20.304558 17372 raft_consensus.cc:2229] T bc57f60850374f6396600477a0d37cac P 5f09a35ebd6f4bb0bbe5799d06ac3a46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:02:20.304986 17371 raft_consensus.cc:2258] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:02:20.305043 17372 raft_consensus.cc:2258] T bc57f60850374f6396600477a0d37cac P 5f09a35ebd6f4bb0bbe5799d06ac3a46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:02:20.306515 17370 tablet_replica.cc:330] T bc57f60850374f6396600477a0d37cac P 5e301f6cfd00456ebf43f313450dd98f: stopping tablet replica
I20260620 23:02:20.307053 17370 raft_consensus.cc:2229] T bc57f60850374f6396600477a0d37cac P 5e301f6cfd00456ebf43f313450dd98f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:02:20.307185 17372 ts_tablet_manager.cc:1905] T bc57f60850374f6396600477a0d37cac P 5f09a35ebd6f4bb0bbe5799d06ac3a46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:02:20.307673 17370 raft_consensus.cc:2258] T bc57f60850374f6396600477a0d37cac P 5e301f6cfd00456ebf43f313450dd98f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:02:20.310083 17370 ts_tablet_manager.cc:1905] T bc57f60850374f6396600477a0d37cac P 5e301f6cfd00456ebf43f313450dd98f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:02:20.311861 17371 ts_tablet_manager.cc:1905] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:02:20.317256 17370 ts_tablet_manager.cc:1918] T bc57f60850374f6396600477a0d37cac P 5e301f6cfd00456ebf43f313450dd98f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:02:20.317523 17370 log.cc:1196] T bc57f60850374f6396600477a0d37cac P 5e301f6cfd00456ebf43f313450dd98f: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/wal/wals/bc57f60850374f6396600477a0d37cac
I20260620 23:02:20.318173 17370 ts_tablet_manager.cc:1939] T bc57f60850374f6396600477a0d37cac P 5e301f6cfd00456ebf43f313450dd98f: Deleting consensus metadata
I20260620 23:02:20.320250 16854 catalog_manager.cc:4841] TS 5e301f6cfd00456ebf43f313450dd98f (127.2.237.1:39915): tablet bc57f60850374f6396600477a0d37cac (table test-table [id=57ca0eea5f884a85a2ef80070776a94c]) successfully deleted
I20260620 23:02:20.322937 17371 ts_tablet_manager.cc:1918] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:02:20.322934 17372 ts_tablet_manager.cc:1918] T bc57f60850374f6396600477a0d37cac P 5f09a35ebd6f4bb0bbe5799d06ac3a46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:02:20.323138 17372 log.cc:1196] T bc57f60850374f6396600477a0d37cac P 5f09a35ebd6f4bb0bbe5799d06ac3a46: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/wal/wals/bc57f60850374f6396600477a0d37cac
I20260620 23:02:20.323237 17371 log.cc:1196] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/wal/wals/bc57f60850374f6396600477a0d37cac
I20260620 23:02:20.323923 17371 ts_tablet_manager.cc:1939] T bc57f60850374f6396600477a0d37cac P 4dc7a8b876e149bd9f773b7cac8a0c30: Deleting consensus metadata
I20260620 23:02:20.324119 17372 ts_tablet_manager.cc:1939] T bc57f60850374f6396600477a0d37cac P 5f09a35ebd6f4bb0bbe5799d06ac3a46: Deleting consensus metadata
I20260620 23:02:20.325852 16856 catalog_manager.cc:4841] TS 4dc7a8b876e149bd9f773b7cac8a0c30 (127.2.237.2:46845): tablet bc57f60850374f6396600477a0d37cac (table test-table [id=57ca0eea5f884a85a2ef80070776a94c]) successfully deleted
I20260620 23:02:20.326042 16855 catalog_manager.cc:4841] TS 5f09a35ebd6f4bb0bbe5799d06ac3a46 (127.2.237.3:44217): tablet bc57f60850374f6396600477a0d37cac (table test-table [id=57ca0eea5f884a85a2ef80070776a94c]) successfully deleted
Jun 20 23:02:20 dist-test-slave-f63b krb5kdc[16825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.1@KRBTEST.COM
I20260620 23:02:20.335237 17342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:20.297527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50680 (local address 127.2.237.1:39915)
0620 23:02:20.297801 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:20.297810 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:20.315221 (+ 17411us) server_negotiation.cc:407] Connection header received
0620 23:02:20.315554 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:20.315561 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:20.315658 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:20.315761 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:20.317872 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:20.319165 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:20.321851 (+ 2686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:20.322483 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:20.322661 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:20.328921 (+ 6260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:20.328934 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:20.328949 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:20.328994 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:20.332030 (+ 3036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:20.332943 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:20.332951 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:20.332953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:20.333095 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:20.333527 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:20.333530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:20.333533 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:20.333882 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:20.334155 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:20.334610 (+ 455us) server_negotiation.cc:299] Negotiation successful
0620 23:02:20.334839 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260620 23:02:20.335954 17054 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:02:20.430114 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 16931
I20260620 23:02:20.448477 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 17065
Jun 20 23:02:20 dist-test-slave-f63b krb5kdc[16825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.2@KRBTEST.COM
I20260620 23:02:20.466722 17343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:20.449573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47706 (local address 127.2.237.2:46845)
0620 23:02:20.449754 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:20.449759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:20.449776 (+ 17us) server_negotiation.cc:407] Connection header received
0620 23:02:20.449836 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:20.449840 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:20.449910 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:20.449994 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:20.451724 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:20.452534 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:20.454474 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:20.454992 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:20.455096 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:20.460542 (+ 5446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:20.460557 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:20.460574 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:20.460611 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:20.463824 (+ 3213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:20.464673 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:20.464681 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:20.464683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:20.464811 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:20.465204 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:20.465207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:20.465210 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:20.465620 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:20.465857 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:20.466172 (+ 315us) server_negotiation.cc:299] Negotiation successful
0620 23:02:20.466362 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260620 23:02:20.467541 17188 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:02:20.581779 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 17065
I20260620 23:02:20.604615 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 17199
Jun 20 23:02:20 dist-test-slave-f63b krb5kdc[16825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.3@KRBTEST.COM
I20260620 23:02:20.619940 17345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:20.605361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52140 (local address 127.2.237.3:44217)
0620 23:02:20.605534 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:20.605538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:20.605558 (+ 20us) server_negotiation.cc:407] Connection header received
0620 23:02:20.605731 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:20.605735 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:20.605810 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:20.605899 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:20.607404 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:20.608151 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:20.609450 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:20.609885 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:20.609984 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:20.614360 (+ 4376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:20.614375 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:20.614391 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:20.614427 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:20.617170 (+ 2743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:20.617962 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:20.617969 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:20.617971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:20.618115 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:20.618442 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:20.618445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:20.618447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:20.618872 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:20.619187 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:20.619395 (+ 208us) server_negotiation.cc:299] Negotiation successful
0620 23:02:20.619570 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 23:02:20.620694 17322 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:02:20.714063 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 17199
I20260620 23:02:20.731226 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 16838
I20260620 23:02:20.732337 16900 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:02:20.864845 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 16838
I20260620 23:02:20.886435 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:41897
--webserver_interface=127.2.237.62
--webserver_port=33987
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:39699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:41897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:02:21.224766 17378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:02:21.225097 17378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:02:21.225232 17378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:02:21.232769 17378 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:02:21.232884 17378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:02:21.232980 17378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:02:21.233091 17378 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:02:21.233182 17378 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:02:21.243430 17378 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:39699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:41897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:41897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=33987
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:02:21.244849 17378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:02:21.246235 17378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:02:21.270586 17384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:21.270622 17387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:21.270527 17385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:02:21.271234 17378 server_base.cc:1034] running on GCE node
I20260620 23:02:21.272289 17378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:02:21.273947 17378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:02:21.275180 17378 hybrid_clock.cc:648] HybridClock initialized: now 1781996541275112 us; error 81 us; skew 500 ppm
I20260620 23:02:21.275594 17378 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:02:21 dist-test-slave-f63b krb5kdc[16825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996541, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:02:21.279327 17378 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 23:02:21.281029 17378 webserver.cc:460] Webserver started at http://127.2.237.62:33987/ using document root <none> and password file <none>
I20260620 23:02:21.281776 17378 fs_manager.cc:359] Metadata directory not provided
I20260620 23:02:21.281891 17378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:02:21.288223 17378 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260620 23:02:21.291571 17394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:21.293073 17378 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 23:02:21.293288 17378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "ebcb4fd8961f46a3a3130026fcc631be"
format_stamp: "Formatted at 2026-06-20 23:02:18 on dist-test-slave-f63b"
server_key: "38f310ff8e6a3c37ed71cd09ff50bd9a"
server_key_iv: "9c55bd0b98672091dc24ee0e91588d91"
server_key_version: "encryptionkey@0"
I20260620 23:02:21.294435 17378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:02:21.312194 17378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:02:21.316695 17378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:02:21.316995 17378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:02:21.336812 17378 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:41897
I20260620 23:02:21.336831 17445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:41897 every 8 connection(s)
I20260620 23:02:21.338486 17378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:02:21.339969 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 17378
I20260620 23:02:21.340797 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:39915
--local_ip_for_outbound_sockets=127.2.237.1
--tserver_master_addrs=127.2.237.62:41897
--webserver_port=46483
--webserver_interface=127.2.237.1
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:39699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:02:21.346132 17446 sys_catalog.cc:263] Verifying existing consensus state
I20260620 23:02:21.350987 17446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be: Bootstrap starting.
I20260620 23:02:21.382324 17446 log.cc:824] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be: Log is configured to *not* fsync() on all Append() calls
I20260620 23:02:21.398679 17446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 23:02:21.399407 17446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be: Bootstrap complete.
I20260620 23:02:21.415112 17446 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcb4fd8961f46a3a3130026fcc631be" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 41897 } }
I20260620 23:02:21.416373 17446 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebcb4fd8961f46a3a3130026fcc631be, State: Initialized, Role: FOLLOWER
I20260620 23:02:21.417177 17446 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcb4fd8961f46a3a3130026fcc631be" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 41897 } }
I20260620 23:02:21.417474 17446 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:02:21.417616 17446 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:02:21.417802 17446 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [term 1 FOLLOWER]: Advancing to term 2
I20260620 23:02:21.423702 17446 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcb4fd8961f46a3a3130026fcc631be" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 41897 } }
I20260620 23:02:21.424396 17446 leader_election.cc:302] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebcb4fd8961f46a3a3130026fcc631be; no voters:
I20260620 23:02:21.425063 17446 leader_election.cc:288] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 23:02:21.425266 17450 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [term 2 FOLLOWER]: Leader election won for term 2
I20260620 23:02:21.426656 17450 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [term 2 LEADER]: Becoming Leader. State: Replica: ebcb4fd8961f46a3a3130026fcc631be, State: Running, Role: LEADER
I20260620 23:02:21.427443 17446 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:02:21.427392 17450 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcb4fd8961f46a3a3130026fcc631be" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 41897 } }
I20260620 23:02:21.430548 17451 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ebcb4fd8961f46a3a3130026fcc631be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcb4fd8961f46a3a3130026fcc631be" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 41897 } } }
I20260620 23:02:21.430475 17452 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebcb4fd8961f46a3a3130026fcc631be. Latest consensus state: current_term: 2 leader_uuid: "ebcb4fd8961f46a3a3130026fcc631be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcb4fd8961f46a3a3130026fcc631be" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 41897 } } }
I20260620 23:02:21.430995 17451 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [sys.catalog]: This master's current role is: LEADER
I20260620 23:02:21.431032 17452 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [sys.catalog]: This master's current role is: LEADER
I20260620 23:02:21.443413 17458 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:02:21.452929 17458 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:02:21.455339 17458 catalog_manager.cc:1259] Loaded cluster ID: 40332be05fe444e38fdd81d798e1fba8
I20260620 23:02:21.455621 17458 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:02:21.460611 17458 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260620 23:02:21.474355 17458 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260620 23:02:21.482403 17458 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:02:21.483959 17458 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be: Loaded TSK: 0
I20260620 23:02:21.485272 17458 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 23:02:21.747227 17448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:02:21.747568 17448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:02:21.747748 17448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:02:21.755376 17448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:02:21.755545 17448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:02:21.755729 17448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 23:02:21.765950 17448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:39699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:39915
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=46483
--tserver_master_addrs=127.2.237.62:41897
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:02:21.767560 17448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:02:21.768994 17448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:02:21.784482 17473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:21.784811 17474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:21.785148 17476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:02:21.785979 17448 server_base.cc:1034] running on GCE node
I20260620 23:02:21.786810 17448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:02:21.788098 17448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:02:21.789392 17448 hybrid_clock.cc:648] HybridClock initialized: now 1781996541789300 us; error 69 us; skew 500 ppm
I20260620 23:02:21.789803 17448 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:02:21 dist-test-slave-f63b krb5kdc[16825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996541, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:02:21.793764 17448 init.cc:377] Logged in from keytab as kudu/127.2.237.1@KRBTEST.COM (short username kudu)
I20260620 23:02:21.795522 17448 webserver.cc:460] Webserver started at http://127.2.237.1:46483/ using document root <none> and password file <none>
I20260620 23:02:21.796286 17448 fs_manager.cc:359] Metadata directory not provided
I20260620 23:02:21.796451 17448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:02:21.802698 17448 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260620 23:02:21.805769 17483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:21.807616 17448 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 23:02:21.807837 17448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "5e301f6cfd00456ebf43f313450dd98f"
format_stamp: "Formatted at 2026-06-20 23:02:18 on dist-test-slave-f63b"
server_key: "ecfcb230876b7c74e38815edb34821c6"
server_key_iv: "c2c8b6161538c8ad32e6fd89fde9925a"
server_key_version: "encryptionkey@0"
I20260620 23:02:21.808950 17448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:02:21.826781 17448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:02:21.832002 17448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:02:21.832398 17448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:02:21.833604 17448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:02:21.835346 17448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:02:21.835490 17448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:21.835646 17448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:02:21.835747 17448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:21.871155 17448 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:39915
I20260620 23:02:21.871189 17595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:39915 every 8 connection(s)
I20260620 23:02:21.872803 17448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 23:02:21.879722 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 17448
I20260620 23:02:21.880626 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:46845
--local_ip_for_outbound_sockets=127.2.237.2
--tserver_master_addrs=127.2.237.62:41897
--webserver_port=44703
--webserver_interface=127.2.237.2
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:39699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 23:02:21 dist-test-slave-f63b krb5kdc[16825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996541, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:02:21.900909 17599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:21.876031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.1:54871)
0620 23:02:21.876884 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:21.876933 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:21.878750 (+ 1817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:21.880710 (+ 1960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:21.880718 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:21.881365 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:02:21.881992 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:21.882011 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:21.884719 (+ 2708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:21.884723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:21.885990 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:21.886003 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:21.887696 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:21.887701 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:21.888077 (+ 376us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:21.888966 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:02:21.889011 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:02:21.892024 (+ 3013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:02:21.895888 (+ 3864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:21.895900 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:21.895920 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:21.896412 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:21.896916 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:21.896921 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:21.896924 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:21.897136 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:21.898065 (+ 929us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:02:21.898074 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:02:21.898451 (+ 377us) client_negotiation.cc:769] Sending connection context
0620 23:02:21.898826 (+ 375us) client_negotiation.cc:240] Negotiation successful
0620 23:02:21.899462 (+ 636us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":350,"threads_started":1}
I20260620 23:02:21.900909 17598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:21.875968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:54871 (local address 127.2.237.62:41897)
0620 23:02:21.876645 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:21.876658 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:21.877100 (+ 442us) server_negotiation.cc:407] Connection header received
0620 23:02:21.879540 (+ 2440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:21.879564 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:21.880124 (+ 560us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:21.880607 (+ 483us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:21.882283 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:21.884519 (+ 2236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:21.886172 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:21.887442 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:21.887687 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:21.892398 (+ 4711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:21.892407 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:21.892425 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:21.892477 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:21.895692 (+ 3215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:21.896580 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:21.896588 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:21.896594 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:21.896773 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:21.897262 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:21.897268 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:21.897273 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:21.897900 (+ 627us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:21.898110 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:21.899048 (+ 938us) server_negotiation.cc:299] Negotiation successful
0620 23:02:21.899472 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":309,"threads_started":1}
I20260620 23:02:21.904626 17596 heartbeater.cc:347] Connected to a master server at 127.2.237.62:41897
I20260620 23:02:21.905040 17596 heartbeater.cc:464] Registering TS with master...
I20260620 23:02:21.906032 17596 heartbeater.cc:511] Master 127.2.237.62:41897 requested a full tablet report, sending...
I20260620 23:02:21.908607 17411 ts_manager.cc:194] Registered new tserver with Master: 5e301f6cfd00456ebf43f313450dd98f (127.2.237.1:39915)
I20260620 23:02:21.910224 17411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.1@KRBTEST.COM'} at 127.2.237.1:54871
W20260620 23:02:22.222072 17600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:02:22.222395 17600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:02:22.222558 17600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:02:22.230463 17600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:02:22.230633 17600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:02:22.230844 17600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 23:02:22.240933 17600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:39699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:46845
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=44703
--tserver_master_addrs=127.2.237.62:41897
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:02:22.242350 17600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:02:22.243894 17600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:02:22.257807 17605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:22.257876 17606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:22.257841 17608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:02:22.259136 17600 server_base.cc:1034] running on GCE node
I20260620 23:02:22.259826 17600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:02:22.260878 17600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:02:22.262118 17600 hybrid_clock.cc:648] HybridClock initialized: now 1781996542262056 us; error 62 us; skew 500 ppm
I20260620 23:02:22.262481 17600 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:02:22 dist-test-slave-f63b krb5kdc[16825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996542, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:02:22.265914 17600 init.cc:377] Logged in from keytab as kudu/127.2.237.2@KRBTEST.COM (short username kudu)
I20260620 23:02:22.267717 17600 webserver.cc:460] Webserver started at http://127.2.237.2:44703/ using document root <none> and password file <none>
I20260620 23:02:22.268563 17600 fs_manager.cc:359] Metadata directory not provided
I20260620 23:02:22.268738 17600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:02:22.274773 17600 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260620 23:02:22.277896 17615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:22.279533 17600 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 23:02:22.279743 17600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "4dc7a8b876e149bd9f773b7cac8a0c30"
format_stamp: "Formatted at 2026-06-20 23:02:19 on dist-test-slave-f63b"
server_key: "6f5759a55668bbbde0588f820884d01b"
server_key_iv: "86f08585b37800801857e2f3debf34d5"
server_key_version: "encryptionkey@0"
I20260620 23:02:22.280813 17600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:02:22.300464 17600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:02:22.304975 17600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:02:22.305282 17600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:02:22.306460 17600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:02:22.307960 17600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:02:22.308084 17600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:22.308216 17600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:02:22.308308 17600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:22.346756 17600 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:46845
I20260620 23:02:22.346822 17727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:46845 every 8 connection(s)
I20260620 23:02:22.348309 17600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 23:02:22.357102 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 17600
I20260620 23:02:22.357838 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:44217
--local_ip_for_outbound_sockets=127.2.237.3
--tserver_master_addrs=127.2.237.62:41897
--webserver_port=36581
--webserver_interface=127.2.237.3
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:39699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 23:02:22 dist-test-slave-f63b krb5kdc[16825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996542, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:02:22.370218 17598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:22.351154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:49765 (local address 127.2.237.62:41897)
0620 23:02:22.351336 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:22.351341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:22.352682 (+ 1341us) server_negotiation.cc:407] Connection header received
0620 23:02:22.354612 (+ 1930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:22.354618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:22.354742 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:22.354857 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:22.356438 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.357501 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.359327 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.360112 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.360265 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:22.363845 (+ 3580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:22.363854 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:22.363862 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:22.363915 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:22.366781 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:22.367780 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:22.367784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:22.367786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:22.367932 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:22.368448 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:22.368451 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:22.368453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:22.368734 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:22.368858 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:22.369779 (+ 921us) server_negotiation.cc:299] Negotiation successful
0620 23:02:22.369979 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 23:02:22.371974 17730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:22.351757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.2:49765)
0620 23:02:22.352510 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:22.352548 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:22.354292 (+ 1744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:22.355125 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:22.355134 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:22.355660 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:02:22.356248 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:22.356267 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.357684 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.357688 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:22.359160 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:22.359174 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.360333 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.360337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:22.360551 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:22.361290 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:02:22.361314 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:02:22.363592 (+ 2278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:02:22.366933 (+ 3341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:22.366945 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:22.366968 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:22.367656 (+ 688us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:22.368049 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:22.368057 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:22.368063 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:22.368334 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:22.368866 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:02:22.368877 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:02:22.369377 (+ 500us) client_negotiation.cc:769] Sending connection context
0620 23:02:22.369708 (+ 331us) client_negotiation.cc:240] Negotiation successful
0620 23:02:22.370352 (+ 644us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":304,"threads_started":1}
I20260620 23:02:22.373631 17728 heartbeater.cc:347] Connected to a master server at 127.2.237.62:41897
I20260620 23:02:22.373976 17728 heartbeater.cc:464] Registering TS with master...
I20260620 23:02:22.374838 17728 heartbeater.cc:511] Master 127.2.237.62:41897 requested a full tablet report, sending...
I20260620 23:02:22.376535 17411 ts_manager.cc:194] Registered new tserver with Master: 4dc7a8b876e149bd9f773b7cac8a0c30 (127.2.237.2:46845)
I20260620 23:02:22.377390 17411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.2@KRBTEST.COM'} at 127.2.237.2:49765
W20260620 23:02:22.695997 17731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:02:22.696346 17731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:02:22.696444 17731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:02:22.704195 17731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:02:22.704378 17731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:02:22.704599 17731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 23:02:22.718024 17731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:39699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:44217
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=36581
--tserver_master_addrs=127.2.237.62:41897
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:02:22.719532 17731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:02:22.720947 17731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:02:22.735347 17737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:22.735256 17736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:22.735229 17739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:02:22.736657 17731 server_base.cc:1034] running on GCE node
I20260620 23:02:22.737334 17731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:02:22.738346 17731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:02:22.739579 17731 hybrid_clock.cc:648] HybridClock initialized: now 1781996542739524 us; error 48 us; skew 500 ppm
I20260620 23:02:22.739928 17731 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:02:22 dist-test-slave-f63b krb5kdc[16825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996542, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:02:22.743337 17731 init.cc:377] Logged in from keytab as kudu/127.2.237.3@KRBTEST.COM (short username kudu)
I20260620 23:02:22.744843 17731 webserver.cc:460] Webserver started at http://127.2.237.3:36581/ using document root <none> and password file <none>
I20260620 23:02:22.745581 17731 fs_manager.cc:359] Metadata directory not provided
I20260620 23:02:22.745741 17731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:02:22.751431 17731 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260620 23:02:22.754241 17746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:22.755707 17731 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260620 23:02:22.755900 17731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46"
format_stamp: "Formatted at 2026-06-20 23:02:19 on dist-test-slave-f63b"
server_key: "3c4f4f339d2cfd06e390ac65416f523f"
server_key_iv: "fc7c874cb24c79f50893da20c674d697"
server_key_version: "encryptionkey@0"
I20260620 23:02:22.756929 17731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:02:22.787813 17731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:02:22.792630 17731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:02:22.792934 17731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:02:22.793972 17731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:02:22.795449 17731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:02:22.795583 17731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:22.795718 17731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:02:22.795815 17731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:22.830355 17731 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:44217
I20260620 23:02:22.830449 17858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:44217 every 8 connection(s)
I20260620 23:02:22.831858 17731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 23:02:22.837843 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 17731
Jun 20 23:02:22 dist-test-slave-f63b krb5kdc[16825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996542, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:02:22.854405 17863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:22.838937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34838 (local address 127.2.237.62:41897)
0620 23:02:22.839765 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:22.839770 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:22.839783 (+ 13us) server_negotiation.cc:407] Connection header received
0620 23:02:22.839821 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:22.839824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:22.839870 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:22.839950 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:22.841650 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.842460 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.844228 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.844684 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.844830 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:22.847945 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:22.847950 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:22.847953 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:22.847994 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:22.851298 (+ 3304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:22.852177 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:22.852182 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:22.852185 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:22.852361 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:22.852808 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:22.852812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:22.852814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:22.853120 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:22.853355 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:22.853729 (+ 374us) server_negotiation.cc:299] Negotiation successful
0620 23:02:22.853984 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":733,"thread_start_us":615,"threads_started":1}
I20260620 23:02:22.854813 17598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:22.834647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:39699 (local address 127.2.237.62:41897)
0620 23:02:22.834818 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:22.834823 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:22.836005 (+ 1182us) server_negotiation.cc:407] Connection header received
0620 23:02:22.838036 (+ 2031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:22.838042 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:22.838121 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:22.838219 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:22.840322 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.841021 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.843205 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.843673 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.843786 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:22.847880 (+ 4094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:22.847887 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:22.847892 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:22.847949 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:22.850995 (+ 3046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:22.851896 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:22.851902 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:22.851908 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:22.852065 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:22.852874 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:22.852879 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:22.852881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:22.853162 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:22.853327 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:22.854289 (+ 962us) server_negotiation.cc:299] Negotiation successful
0620 23:02:22.854447 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260620 23:02:22.856178 17861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:22.835118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.3:39699)
0620 23:02:22.835820 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:22.835860 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:22.837673 (+ 1813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:22.838524 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:22.838533 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:22.839316 (+ 783us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:02:22.840148 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:22.840177 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.841174 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.841179 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:22.842959 (+ 1780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:22.842974 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.843923 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.843932 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:22.844251 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:22.844951 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:02:22.844974 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:02:22.847625 (+ 2651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:02:22.851164 (+ 3539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:22.851176 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:22.851196 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:22.851743 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:22.852185 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:22.852190 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:22.852194 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:22.852493 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:22.853330 (+ 837us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:02:22.853339 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:02:22.853795 (+ 456us) client_negotiation.cc:769] Sending connection context
0620 23:02:22.854158 (+ 363us) client_negotiation.cc:240] Negotiation successful
0620 23:02:22.854799 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":272,"threads_started":1}
I20260620 23:02:22.857717 17859 heartbeater.cc:347] Connected to a master server at 127.2.237.62:41897
I20260620 23:02:22.857998 17859 heartbeater.cc:464] Registering TS with master...
I20260620 23:02:22.858757 17859 heartbeater.cc:511] Master 127.2.237.62:41897 requested a full tablet report, sending...
I20260620 23:02:22.860098 17411 ts_manager.cc:194] Registered new tserver with Master: 5f09a35ebd6f4bb0bbe5799d06ac3a46 (127.2.237.3:44217)
I20260620 23:02:22.861032 17411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.3@KRBTEST.COM'} at 127.2.237.3:39699
I20260620 23:02:22.870143 2996 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:02:22.878058 17598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:22.871153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34852 (local address 127.2.237.62:41897)
0620 23:02:22.871383 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:22.871403 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:22.871909 (+ 506us) server_negotiation.cc:407] Connection header received
0620 23:02:22.872137 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:22.872140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:22.872280 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:22.872414 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:02:22.872959 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.873808 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.875913 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.876326 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.876432 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:22.877016 (+ 584us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:02:22.877491 (+ 475us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:02:22.877601 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:22.877705 (+ 104us) server_negotiation.cc:299] Negotiation successful
0620 23:02:22.877773 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260620 23:02:22.879128 17411 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:34852:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:02:22.886951 17411 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:02:22.914331 17596 heartbeater.cc:503] Master 127.2.237.62:41897 was elected leader, sending a full tablet report...
I20260620 23:02:22.925707 17872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:22.899896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47708 (local address 127.2.237.2:46845)
0620 23:02:22.915335 (+ 15439us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:22.915343 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:22.915375 (+ 32us) server_negotiation.cc:407] Connection header received
0620 23:02:22.915492 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:22.915501 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:22.915819 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:22.917112 (+ 1293us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:02:22.917405 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.919577 (+ 2172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.922725 (+ 3148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.924397 (+ 1672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.924529 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:22.924737 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:22.925180 (+ 443us) server_negotiation.cc:299] Negotiation successful
0620 23:02:22.925362 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15280,"thread_start_us":15137,"threads_started":1}
I20260620 23:02:22.925752 17870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:22.899556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:46845 (local address 127.0.0.1:47708)
0620 23:02:22.913046 (+ 13490us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:22.913074 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:22.913265 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:22.916167 (+ 2902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:22.916179 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:22.916224 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:02:22.916812 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:22.916829 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.919831 (+ 3002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.919840 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:22.922460 (+ 2620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:22.922472 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.924617 (+ 2145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.924624 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:22.924876 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:22.924942 (+ 66us) client_negotiation.cc:769] Sending connection context
0620 23:02:22.925091 (+ 149us) client_negotiation.cc:240] Negotiation successful
0620 23:02:22.925317 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12959,"mutex_wait_us":52,"thread_start_us":13212,"threads_started":1}
I20260620 23:02:22.927615 17869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:22.899137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:44217 (local address 127.0.0.1:52146)
0620 23:02:22.900163 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:22.900215 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:22.900407 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:22.920483 (+ 20076us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:22.920487 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:22.920509 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:02:22.920928 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:22.920938 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.922778 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.922785 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:22.924856 (+ 2071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:22.924868 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.926743 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.926750 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:22.926927 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:22.926961 (+ 34us) client_negotiation.cc:769] Sending connection context
0620 23:02:22.927073 (+ 112us) client_negotiation.cc:240] Negotiation successful
0620 23:02:22.927226 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":783,"mutex_wait_us":3,"thread_start_us":657,"threads_started":1}
I20260620 23:02:22.927661 17871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:22.899517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52146 (local address 127.2.237.3:44217)
0620 23:02:22.919758 (+ 20241us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:22.919770 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:22.919812 (+ 42us) server_negotiation.cc:407] Connection header received
0620 23:02:22.919929 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:22.919940 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:22.920264 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:22.920501 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:02:22.921129 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.922621 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.925022 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.926474 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.926642 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:22.926877 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:22.927126 (+ 249us) server_negotiation.cc:299] Negotiation successful
0620 23:02:22.927340 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20060,"thread_start_us":19891,"threads_started":1}
I20260620 23:02:22.930474 17873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:22.900764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:39915 (local address 127.0.0.1:50686)
0620 23:02:22.921060 (+ 20296us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:22.921085 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:22.921219 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:22.921891 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:22.921896 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:22.921921 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:02:22.922415 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:22.922434 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.924250 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.924255 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:22.926563 (+ 2308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:22.926577 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.929646 (+ 3069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.929652 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:22.929860 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:22.929891 (+ 31us) client_negotiation.cc:769] Sending connection context
0620 23:02:22.929985 (+ 94us) client_negotiation.cc:240] Negotiation successful
0620 23:02:22.930105 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20127,"mutex_wait_us":1,"thread_start_us":19058,"threads_started":1}
I20260620 23:02:22.930320 17663 tablet_service.cc:1467] Processing CreateTablet for tablet 5b7b9d2c46514007a68a2bf50d14558b (DEFAULT_TABLE table=test-table [id=46840ebb2b4f470da06867c1570987bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:02:22.931234 17874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:22.901152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50686 (local address 127.2.237.1:39915)
0620 23:02:22.914881 (+ 13729us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:22.914892 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:22.921189 (+ 6297us) server_negotiation.cc:407] Connection header received
0620 23:02:22.921372 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:22.921382 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:22.921667 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:22.921884 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:02:22.922657 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.924090 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.926756 (+ 2666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.929109 (+ 2353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.929214 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:22.929430 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:22.930750 (+ 1320us) server_negotiation.cc:299] Negotiation successful
0620 23:02:22.930912 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12682,"thread_start_us":13372,"threads_started":1}
I20260620 23:02:22.931874 17663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b7b9d2c46514007a68a2bf50d14558b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:02:22.931751 17794 tablet_service.cc:1467] Processing CreateTablet for tablet 5b7b9d2c46514007a68a2bf50d14558b (DEFAULT_TABLE table=test-table [id=46840ebb2b4f470da06867c1570987bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:02:22.932958 17794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b7b9d2c46514007a68a2bf50d14558b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:02:22.935156 17531 tablet_service.cc:1467] Processing CreateTablet for tablet 5b7b9d2c46514007a68a2bf50d14558b (DEFAULT_TABLE table=test-table [id=46840ebb2b4f470da06867c1570987bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:02:22.936893 17531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b7b9d2c46514007a68a2bf50d14558b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:02:22.945708 17875 tablet_bootstrap.cc:492] T 5b7b9d2c46514007a68a2bf50d14558b P 5f09a35ebd6f4bb0bbe5799d06ac3a46: Bootstrap starting.
I20260620 23:02:22.949584 17875 tablet_bootstrap.cc:654] T 5b7b9d2c46514007a68a2bf50d14558b P 5f09a35ebd6f4bb0bbe5799d06ac3a46: Neither blocks nor log segments found. Creating new log.
I20260620 23:02:22.951014 17875 log.cc:824] T 5b7b9d2c46514007a68a2bf50d14558b P 5f09a35ebd6f4bb0bbe5799d06ac3a46: Log is configured to *not* fsync() on all Append() calls
I20260620 23:02:22.951946 17876 tablet_bootstrap.cc:492] T 5b7b9d2c46514007a68a2bf50d14558b P 4dc7a8b876e149bd9f773b7cac8a0c30: Bootstrap starting.
I20260620 23:02:22.955663 17876 tablet_bootstrap.cc:654] T 5b7b9d2c46514007a68a2bf50d14558b P 4dc7a8b876e149bd9f773b7cac8a0c30: Neither blocks nor log segments found. Creating new log.
I20260620 23:02:22.955776 17878 tablet_bootstrap.cc:492] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f: Bootstrap starting.
I20260620 23:02:22.956005 17875 tablet_bootstrap.cc:492] T 5b7b9d2c46514007a68a2bf50d14558b P 5f09a35ebd6f4bb0bbe5799d06ac3a46: No bootstrap required, opened a new log
I20260620 23:02:22.956305 17875 ts_tablet_manager.cc:1397] T 5b7b9d2c46514007a68a2bf50d14558b P 5f09a35ebd6f4bb0bbe5799d06ac3a46: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260620 23:02:22.957839 17876 log.cc:824] T 5b7b9d2c46514007a68a2bf50d14558b P 4dc7a8b876e149bd9f773b7cac8a0c30: Log is configured to *not* fsync() on all Append() calls
I20260620 23:02:22.959467 17878 tablet_bootstrap.cc:654] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f: Neither blocks nor log segments found. Creating new log.
I20260620 23:02:22.960757 17878 log.cc:824] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f: Log is configured to *not* fsync() on all Append() calls
I20260620 23:02:22.961488 17876 tablet_bootstrap.cc:492] T 5b7b9d2c46514007a68a2bf50d14558b P 4dc7a8b876e149bd9f773b7cac8a0c30: No bootstrap required, opened a new log
I20260620 23:02:22.961812 17876 ts_tablet_manager.cc:1397] T 5b7b9d2c46514007a68a2bf50d14558b P 4dc7a8b876e149bd9f773b7cac8a0c30: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260620 23:02:22.964134 17878 tablet_bootstrap.cc:492] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f: No bootstrap required, opened a new log
I20260620 23:02:22.964428 17878 ts_tablet_manager.cc:1397] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260620 23:02:22.965500 17875 raft_consensus.cc:348] T 5b7b9d2c46514007a68a2bf50d14558b P 5f09a35ebd6f4bb0bbe5799d06ac3a46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44217 } } peers { permanent_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46845 } } peers { permanent_uuid: "5e301f6cfd00456ebf43f313450dd98f" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 39915 } }
I20260620 23:02:22.965917 17875 raft_consensus.cc:374] T 5b7b9d2c46514007a68a2bf50d14558b P 5f09a35ebd6f4bb0bbe5799d06ac3a46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:02:22.966063 17875 raft_consensus.cc:729] T 5b7b9d2c46514007a68a2bf50d14558b P 5f09a35ebd6f4bb0bbe5799d06ac3a46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f09a35ebd6f4bb0bbe5799d06ac3a46, State: Initialized, Role: FOLLOWER
I20260620 23:02:22.966794 17875 consensus_queue.cc:260] T 5b7b9d2c46514007a68a2bf50d14558b P 5f09a35ebd6f4bb0bbe5799d06ac3a46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44217 } } peers { permanent_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46845 } } peers { permanent_uuid: "5e301f6cfd00456ebf43f313450dd98f" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 39915 } }
I20260620 23:02:22.967962 17859 heartbeater.cc:503] Master 127.2.237.62:41897 was elected leader, sending a full tablet report...
I20260620 23:02:22.968721 17875 ts_tablet_manager.cc:1428] T 5b7b9d2c46514007a68a2bf50d14558b P 5f09a35ebd6f4bb0bbe5799d06ac3a46: Time spent starting tablet: real 0.012s user 0.007s sys 0.004s
I20260620 23:02:22.972764 17876 raft_consensus.cc:348] T 5b7b9d2c46514007a68a2bf50d14558b P 4dc7a8b876e149bd9f773b7cac8a0c30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44217 } } peers { permanent_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46845 } } peers { permanent_uuid: "5e301f6cfd00456ebf43f313450dd98f" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 39915 } }
I20260620 23:02:22.973258 17876 raft_consensus.cc:374] T 5b7b9d2c46514007a68a2bf50d14558b P 4dc7a8b876e149bd9f773b7cac8a0c30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:02:22.973379 17876 raft_consensus.cc:729] T 5b7b9d2c46514007a68a2bf50d14558b P 4dc7a8b876e149bd9f773b7cac8a0c30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dc7a8b876e149bd9f773b7cac8a0c30, State: Initialized, Role: FOLLOWER
I20260620 23:02:22.974037 17876 consensus_queue.cc:260] T 5b7b9d2c46514007a68a2bf50d14558b P 4dc7a8b876e149bd9f773b7cac8a0c30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44217 } } peers { permanent_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46845 } } peers { permanent_uuid: "5e301f6cfd00456ebf43f313450dd98f" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 39915 } }
I20260620 23:02:22.975416 17878 raft_consensus.cc:348] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44217 } } peers { permanent_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46845 } } peers { permanent_uuid: "5e301f6cfd00456ebf43f313450dd98f" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 39915 } }
I20260620 23:02:22.976011 17878 raft_consensus.cc:374] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:02:22.976131 17878 raft_consensus.cc:729] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e301f6cfd00456ebf43f313450dd98f, State: Initialized, Role: FOLLOWER
I20260620 23:02:22.976933 17878 consensus_queue.cc:260] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44217 } } peers { permanent_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46845 } } peers { permanent_uuid: "5e301f6cfd00456ebf43f313450dd98f" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 39915 } }
I20260620 23:02:22.979303 17878 ts_tablet_manager.cc:1428] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f: Time spent starting tablet: real 0.015s user 0.011s sys 0.005s
I20260620 23:02:22.981178 17728 heartbeater.cc:503] Master 127.2.237.62:41897 was elected leader, sending a full tablet report...
I20260620 23:02:22.982232 17876 ts_tablet_manager.cc:1428] T 5b7b9d2c46514007a68a2bf50d14558b P 4dc7a8b876e149bd9f773b7cac8a0c30: Time spent starting tablet: real 0.020s user 0.015s sys 0.004s
I20260620 23:02:22.985121 17883 raft_consensus.cc:482] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:02:22.985423 17883 raft_consensus.cc:504] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44217 } } peers { permanent_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46845 } } peers { permanent_uuid: "5e301f6cfd00456ebf43f313450dd98f" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 39915 } }
I20260620 23:02:22.987143 17883 leader_election.cc:288] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f09a35ebd6f4bb0bbe5799d06ac3a46 (127.2.237.3:44217), 4dc7a8b876e149bd9f773b7cac8a0c30 (127.2.237.2:46845)
I20260620 23:02:22.994030 17872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:22.988020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:55339 (local address 127.2.237.2:46845)
0620 23:02:22.988202 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:22.988207 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:22.988522 (+ 315us) server_negotiation.cc:407] Connection header received
0620 23:02:22.988822 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:22.988827 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:22.988886 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:22.988991 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:02:22.989609 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.990340 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.992261 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.993063 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.993224 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:22.993328 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:22.993627 (+ 299us) server_negotiation.cc:299] Negotiation successful
0620 23:02:22.993772 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260620 23:02:22.994117 17884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:22.987524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:46845 (local address 127.2.237.1:55339)
0620 23:02:22.988391 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:22.988417 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:22.988665 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:22.989029 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:22.989035 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:22.989067 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:02:22.989467 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:22.989477 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.990479 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.990485 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:22.992106 (+ 1621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:22.992119 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.993292 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.993298 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:22.993451 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:22.993476 (+ 25us) client_negotiation.cc:769] Sending connection context
0620 23:02:22.993599 (+ 123us) client_negotiation.cc:240] Negotiation successful
0620 23:02:22.993734 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":838,"threads_started":1}
I20260620 23:02:22.994494 17871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:22.987748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:50593 (local address 127.2.237.3:44217)
0620 23:02:22.987976 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:22.987983 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:22.989381 (+ 1398us) server_negotiation.cc:407] Connection header received
0620 23:02:22.989573 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:22.989578 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:22.989647 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:22.989744 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:02:22.990231 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.991183 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.992825 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.993601 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.993754 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:22.993867 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:22.994134 (+ 267us) server_negotiation.cc:299] Negotiation successful
0620 23:02:22.994255 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 23:02:22.995047 17885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:22.987525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:44217 (local address 127.2.237.1:50593)
0620 23:02:22.989270 (+ 1745us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:22.989284 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:22.989442 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:22.989767 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:22.989771 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:22.989794 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:02:22.990082 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:22.990091 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.991322 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.991327 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:22.992667 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:22.992678 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:22.993822 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:22.993828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:22.993979 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:22.994003 (+ 24us) client_negotiation.cc:769] Sending connection context
0620 23:02:22.994112 (+ 109us) client_negotiation.cc:240] Negotiation successful
0620 23:02:22.994559 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1626,"mutex_wait_us":131,"thread_start_us":1266,"threads_started":1}
I20260620 23:02:22.995337 17683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b7b9d2c46514007a68a2bf50d14558b" candidate_uuid: "5e301f6cfd00456ebf43f313450dd98f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" is_pre_election: true
I20260620 23:02:22.995880 17683 raft_consensus.cc:2454] T 5b7b9d2c46514007a68a2bf50d14558b P 4dc7a8b876e149bd9f773b7cac8a0c30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e301f6cfd00456ebf43f313450dd98f in term 0.
I20260620 23:02:22.996133 17814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b7b9d2c46514007a68a2bf50d14558b" candidate_uuid: "5e301f6cfd00456ebf43f313450dd98f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46" is_pre_election: true
I20260620 23:02:22.996605 17486 leader_election.cc:302] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dc7a8b876e149bd9f773b7cac8a0c30, 5e301f6cfd00456ebf43f313450dd98f; no voters:
I20260620 23:02:22.996661 17814 raft_consensus.cc:2454] T 5b7b9d2c46514007a68a2bf50d14558b P 5f09a35ebd6f4bb0bbe5799d06ac3a46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e301f6cfd00456ebf43f313450dd98f in term 0.
I20260620 23:02:22.997087 17883 raft_consensus.cc:2781] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:02:22.997262 17883 raft_consensus.cc:482] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:02:22.997428 17883 raft_consensus.cc:3037] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:02:22.999845 17883 raft_consensus.cc:504] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44217 } } peers { permanent_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46845 } } peers { permanent_uuid: "5e301f6cfd00456ebf43f313450dd98f" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 39915 } }
I20260620 23:02:23.000452 17883 leader_election.cc:288] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [CANDIDATE]: Term 1 election: Requested vote from peers 5f09a35ebd6f4bb0bbe5799d06ac3a46 (127.2.237.3:44217), 4dc7a8b876e149bd9f773b7cac8a0c30 (127.2.237.2:46845)
I20260620 23:02:23.000962 17814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b7b9d2c46514007a68a2bf50d14558b" candidate_uuid: "5e301f6cfd00456ebf43f313450dd98f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46"
I20260620 23:02:23.001181 17814 raft_consensus.cc:3037] T 5b7b9d2c46514007a68a2bf50d14558b P 5f09a35ebd6f4bb0bbe5799d06ac3a46 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:02:23.001201 17683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b7b9d2c46514007a68a2bf50d14558b" candidate_uuid: "5e301f6cfd00456ebf43f313450dd98f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30"
I20260620 23:02:23.001456 17683 raft_consensus.cc:3037] T 5b7b9d2c46514007a68a2bf50d14558b P 4dc7a8b876e149bd9f773b7cac8a0c30 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:02:23.004347 17814 raft_consensus.cc:2454] T 5b7b9d2c46514007a68a2bf50d14558b P 5f09a35ebd6f4bb0bbe5799d06ac3a46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e301f6cfd00456ebf43f313450dd98f in term 1.
I20260620 23:02:23.004542 17683 raft_consensus.cc:2454] T 5b7b9d2c46514007a68a2bf50d14558b P 4dc7a8b876e149bd9f773b7cac8a0c30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e301f6cfd00456ebf43f313450dd98f in term 1.
I20260620 23:02:23.004938 17485 leader_election.cc:302] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e301f6cfd00456ebf43f313450dd98f, 5f09a35ebd6f4bb0bbe5799d06ac3a46; no voters:
I20260620 23:02:23.005293 17883 raft_consensus.cc:2781] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:02:23.005831 17883 raft_consensus.cc:686] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [term 1 LEADER]: Becoming Leader. State: Replica: 5e301f6cfd00456ebf43f313450dd98f, State: Running, Role: LEADER
I20260620 23:02:23.006533 17883 consensus_queue.cc:237] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44217 } } peers { permanent_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46845 } } peers { permanent_uuid: "5e301f6cfd00456ebf43f313450dd98f" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 39915 } }
I20260620 23:02:23.012004 17410 catalog_manager.cc:5495] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e301f6cfd00456ebf43f313450dd98f (127.2.237.1). New cstate: current_term: 1 leader_uuid: "5e301f6cfd00456ebf43f313450dd98f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e301f6cfd00456ebf43f313450dd98f" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 39915 } health_report { overall_health: HEALTHY } } }
I20260620 23:02:23.046380 17874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:23.040443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50690 (local address 127.2.237.1:39915)
0620 23:02:23.040688 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:23.040695 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:23.040722 (+ 27us) server_negotiation.cc:407] Connection header received
0620 23:02:23.040849 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:23.040853 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:23.040949 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:23.041132 (+ 183us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:02:23.041541 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:23.042301 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:23.044007 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:23.044450 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:23.044605 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:23.045174 (+ 569us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:02:23.045724 (+ 550us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:02:23.045910 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:23.045986 (+ 76us) server_negotiation.cc:299] Negotiation successful
0620 23:02:23.046074 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260620 23:02:23.055482 17814 raft_consensus.cc:1261] T 5b7b9d2c46514007a68a2bf50d14558b P 5f09a35ebd6f4bb0bbe5799d06ac3a46 [term 1 FOLLOWER]: Refusing update from remote peer 5e301f6cfd00456ebf43f313450dd98f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:02:23.055490 17683 raft_consensus.cc:1261] T 5b7b9d2c46514007a68a2bf50d14558b P 4dc7a8b876e149bd9f773b7cac8a0c30 [term 1 FOLLOWER]: Refusing update from remote peer 5e301f6cfd00456ebf43f313450dd98f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:02:23.056723 17886 consensus_queue.cc:1035] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:02:23.057098 17883 consensus_queue.cc:1035] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:02:23.074942 17891 mvcc.cc:205] Tried to move back new op lower bound from 7299057840343326720 to 7299057840160296960. Current Snapshot: MvccSnapshot[applied={T|T < 7299057840343326720}]
I20260620 23:02:23.085668 17411 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:34852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:02:23.086023 17411 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:34852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:02:23.089812 17411 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be: Sending DeleteTablet for 3 replicas of tablet 5b7b9d2c46514007a68a2bf50d14558b
I20260620 23:02:23.091229 17794 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b7b9d2c46514007a68a2bf50d14558b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:02:23 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:52146
I20260620 23:02:23.091476 17663 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b7b9d2c46514007a68a2bf50d14558b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:02:23 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:47708
I20260620 23:02:23.091753 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 17448
I20260620 23:02:23.092373 17531 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b7b9d2c46514007a68a2bf50d14558b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:02:23 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:50686
I20260620 23:02:23.092806 17899 tablet_replica.cc:330] T 5b7b9d2c46514007a68a2bf50d14558b P 5f09a35ebd6f4bb0bbe5799d06ac3a46: stopping tablet replica
I20260620 23:02:23.093281 17899 raft_consensus.cc:2229] T 5b7b9d2c46514007a68a2bf50d14558b P 5f09a35ebd6f4bb0bbe5799d06ac3a46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:02:23.093847 17899 raft_consensus.cc:2258] T 5b7b9d2c46514007a68a2bf50d14558b P 5f09a35ebd6f4bb0bbe5799d06ac3a46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:02:23.094900 17901 tablet_replica.cc:330] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f: stopping tablet replica
I20260620 23:02:23.095093 17900 tablet_replica.cc:330] T 5b7b9d2c46514007a68a2bf50d14558b P 4dc7a8b876e149bd9f773b7cac8a0c30: stopping tablet replica
I20260620 23:02:23.097180 17900 raft_consensus.cc:2229] T 5b7b9d2c46514007a68a2bf50d14558b P 4dc7a8b876e149bd9f773b7cac8a0c30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:02:23.097429 17899 ts_tablet_manager.cc:1905] T 5b7b9d2c46514007a68a2bf50d14558b P 5f09a35ebd6f4bb0bbe5799d06ac3a46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:02:23.097847 17900 raft_consensus.cc:2258] T 5b7b9d2c46514007a68a2bf50d14558b P 4dc7a8b876e149bd9f773b7cac8a0c30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:02:23.098889 17901 raft_consensus.cc:2229] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:02:23.101497 17900 ts_tablet_manager.cc:1905] T 5b7b9d2c46514007a68a2bf50d14558b P 4dc7a8b876e149bd9f773b7cac8a0c30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:02:23.105818 17899 ts_tablet_manager.cc:1918] T 5b7b9d2c46514007a68a2bf50d14558b P 5f09a35ebd6f4bb0bbe5799d06ac3a46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:02:23.106108 17899 log.cc:1196] T 5b7b9d2c46514007a68a2bf50d14558b P 5f09a35ebd6f4bb0bbe5799d06ac3a46: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/wal/wals/5b7b9d2c46514007a68a2bf50d14558b
I20260620 23:02:23.106858 17899 ts_tablet_manager.cc:1939] T 5b7b9d2c46514007a68a2bf50d14558b P 5f09a35ebd6f4bb0bbe5799d06ac3a46: Deleting consensus metadata
I20260620 23:02:23.109119 17396 catalog_manager.cc:4841] TS 5f09a35ebd6f4bb0bbe5799d06ac3a46 (127.2.237.3:44217): tablet 5b7b9d2c46514007a68a2bf50d14558b (table test-table [id=46840ebb2b4f470da06867c1570987bc]) successfully deleted
I20260620 23:02:23.109529 17900 ts_tablet_manager.cc:1918] T 5b7b9d2c46514007a68a2bf50d14558b P 4dc7a8b876e149bd9f773b7cac8a0c30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:02:23.109791 17900 log.cc:1196] T 5b7b9d2c46514007a68a2bf50d14558b P 4dc7a8b876e149bd9f773b7cac8a0c30: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/wal/wals/5b7b9d2c46514007a68a2bf50d14558b
I20260620 23:02:23.110001 17901 raft_consensus.cc:2258] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:02:23.110775 17900 ts_tablet_manager.cc:1939] T 5b7b9d2c46514007a68a2bf50d14558b P 4dc7a8b876e149bd9f773b7cac8a0c30: Deleting consensus metadata
I20260620 23:02:23.112651 17901 ts_tablet_manager.cc:1905] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:02:23.112819 17397 catalog_manager.cc:4841] TS 4dc7a8b876e149bd9f773b7cac8a0c30 (127.2.237.2:46845): tablet 5b7b9d2c46514007a68a2bf50d14558b (table test-table [id=46840ebb2b4f470da06867c1570987bc]) successfully deleted
I20260620 23:02:23.113118 17874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:23.093043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50702 (local address 127.2.237.1:39915)
0620 23:02:23.093234 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:23.093241 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:23.093264 (+ 23us) server_negotiation.cc:407] Connection header received
0620 23:02:23.093332 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:23.093337 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:23.093423 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:23.093544 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:23.095418 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:23.096788 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:23.098799 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:23.099620 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:23.099773 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:23.103605 (+ 3832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:23.103623 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:23.103643 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:23.103693 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:23.108131 (+ 4438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:23.109223 (+ 1092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:23.109233 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:23.109237 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:23.109438 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:23.110135 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:23.110141 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:23.110145 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:23.110626 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:23.111030 (+ 404us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:23.112259 (+ 1229us) server_negotiation.cc:299] Negotiation successful
0620 23:02:23.112535 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260620 23:02:23.114125 17591 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:02:23.213817 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 17448
W20260620 23:02:23.232708 17395 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.237.1:39915 (error 108)
I20260620 23:02:23.233009 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 17600
I20260620 23:02:23.234759 17873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:23.234241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:39915 (local address 127.0.0.1:50710)
0620 23:02:23.234396 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:23.234559 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:39915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260620 23:02:23.235067 17395 catalog_manager.cc:4584] TS 5e301f6cfd00456ebf43f313450dd98f (127.2.237.1:39915): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5b7b9d2c46514007a68a2bf50d14558b: Network error: Client connection negotiation failed: client connection to 127.2.237.1:39915: connect: Connection refused (error 111)
I20260620 23:02:23.251639 17872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:23.233915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47724 (local address 127.2.237.2:46845)
0620 23:02:23.234125 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:23.234130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:23.234148 (+ 18us) server_negotiation.cc:407] Connection header received
0620 23:02:23.234286 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:23.234312 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:23.234399 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:23.234522 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:23.236186 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:23.236915 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:23.238279 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:23.238760 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:23.238910 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:23.245689 (+ 6779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:23.245706 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:23.245723 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:23.245762 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:23.248617 (+ 2855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:23.249444 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:23.249452 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:23.249455 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:23.249586 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:23.249996 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:23.250006 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:23.250008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:23.250392 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:23.250728 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:23.251061 (+ 333us) server_negotiation.cc:299] Negotiation successful
0620 23:02:23.251295 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260620 23:02:23.252352 17723 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:02:23.288830 17873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:23.288226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:39915 (local address 127.0.0.1:50724)
0620 23:02:23.288564 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:23.288667 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:39915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196}
I20260620 23:02:23.289943 17873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:23.289612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:39915 (local address 127.0.0.1:50728)
0620 23:02:23.289748 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:23.289818 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:39915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260620 23:02:23.346827 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 17600
I20260620 23:02:23.350911 17873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:23.350301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:39915 (local address 127.0.0.1:50732)
0620 23:02:23.350529 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:23.350698 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:39915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260620 23:02:23.352214 17873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:23.351735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:39915 (local address 127.0.0.1:50740)
0620 23:02:23.351926 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:23.352030 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:39915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260620 23:02:23.366039 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 17731
I20260620 23:02:23.380360 17871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:23.366939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52152 (local address 127.2.237.3:44217)
0620 23:02:23.367166 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:23.367170 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:23.367192 (+ 22us) server_negotiation.cc:407] Connection header received
0620 23:02:23.367292 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:23.367296 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:23.367363 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:23.367457 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:23.368884 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:23.369757 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:23.371257 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:23.371715 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:23.371855 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:23.374534 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:23.374555 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:23.374631 (+ 76us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:23.374689 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:23.377612 (+ 2923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:23.378301 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:23.378309 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:23.378311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:23.378453 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:23.378883 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:23.378887 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:23.378889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:23.379273 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:23.379520 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:23.379774 (+ 254us) server_negotiation.cc:299] Negotiation successful
0620 23:02:23.379955 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260620 23:02:23.381072 17854 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:02:23.433117 17873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:23.432618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:39915 (local address 127.0.0.1:50756)
0620 23:02:23.432858 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:23.432954 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:39915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260620 23:02:23.434417 17873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:23.433961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:39915 (local address 127.0.0.1:50764)
0620 23:02:23.434197 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:23.434298 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:39915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260620 23:02:23.474639 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 17731
I20260620 23:02:23.491696 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 17378
I20260620 23:02:23.492921 17441 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:02:23.609772 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 17378
I20260620 23:02:23.609880 17873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:23.608911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:39915 (local address 127.0.0.1:50768)
0620 23:02:23.609111 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:23.609225 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:39915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260620 23:02:23.632509 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:41897
--webserver_interface=127.2.237.62
--webserver_port=33987
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:39699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:41897
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:02:23.978888 17907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:02:23.979233 17907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:02:23.979327 17907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:02:23.986907 17907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:02:23.987046 17907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:02:23.987092 17907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:02:23.987143 17907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:02:23.987190 17907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:02:23.997772 17907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:39699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:41897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:41897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=33987
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:02:23.999245 17907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:02:24.000532 17907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:02:24.014282 17915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:24.014375 17913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:24.014617 17912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:02:24.015353 17907 server_base.cc:1034] running on GCE node
I20260620 23:02:24.016232 17907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:02:24.017798 17907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:02:24.019004 17907 hybrid_clock.cc:648] HybridClock initialized: now 1781996544018951 us; error 42 us; skew 500 ppm
I20260620 23:02:24.019397 17907 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:02:24 dist-test-slave-f63b krb5kdc[16825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996544, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:02:24.022871 17907 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 23:02:24.024577 17907 webserver.cc:460] Webserver started at http://127.2.237.62:33987/ using document root <none> and password file <none>
I20260620 23:02:24.025329 17907 fs_manager.cc:359] Metadata directory not provided
I20260620 23:02:24.025468 17907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:02:24.031409 17907 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260620 23:02:24.034653 17922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:24.036230 17907 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260620 23:02:24.036423 17907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "ebcb4fd8961f46a3a3130026fcc631be"
format_stamp: "Formatted at 2026-06-20 23:02:18 on dist-test-slave-f63b"
server_key: "38f310ff8e6a3c37ed71cd09ff50bd9a"
server_key_iv: "9c55bd0b98672091dc24ee0e91588d91"
server_key_version: "encryptionkey@0"
I20260620 23:02:24.037474 17907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:02:24.066213 17907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:02:24.070673 17907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:02:24.070999 17907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:02:24.090891 17907 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:41897
I20260620 23:02:24.090910 17973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:41897 every 8 connection(s)
I20260620 23:02:24.092516 17907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:02:24.098862 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 17907
I20260620 23:02:24.099936 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:39915
--local_ip_for_outbound_sockets=127.2.237.1
--tserver_master_addrs=127.2.237.62:41897
--webserver_port=46483
--webserver_interface=127.2.237.1
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:39699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:02:24.101517 17974 sys_catalog.cc:263] Verifying existing consensus state
I20260620 23:02:24.107321 17974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be: Bootstrap starting.
I20260620 23:02:24.128954 17974 log.cc:824] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be: Log is configured to *not* fsync() on all Append() calls
I20260620 23:02:24.146297 17974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 23:02:24.146934 17974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be: Bootstrap complete.
I20260620 23:02:24.156332 17974 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcb4fd8961f46a3a3130026fcc631be" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 41897 } }
I20260620 23:02:24.157290 17974 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebcb4fd8961f46a3a3130026fcc631be, State: Initialized, Role: FOLLOWER
I20260620 23:02:24.158008 17974 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcb4fd8961f46a3a3130026fcc631be" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 41897 } }
I20260620 23:02:24.158252 17974 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:02:24.158406 17974 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:02:24.158648 17974 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [term 2 FOLLOWER]: Advancing to term 3
I20260620 23:02:24.162737 17974 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcb4fd8961f46a3a3130026fcc631be" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 41897 } }
I20260620 23:02:24.163254 17974 leader_election.cc:302] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebcb4fd8961f46a3a3130026fcc631be; no voters:
I20260620 23:02:24.163954 17974 leader_election.cc:288] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [CANDIDATE]: Term 3 election: Requested vote from peers
I20260620 23:02:24.164098 17978 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [term 3 FOLLOWER]: Leader election won for term 3
I20260620 23:02:24.165313 17978 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [term 3 LEADER]: Becoming Leader. State: Replica: ebcb4fd8961f46a3a3130026fcc631be, State: Running, Role: LEADER
I20260620 23:02:24.165946 17978 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcb4fd8961f46a3a3130026fcc631be" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 41897 } }
I20260620 23:02:24.166288 17974 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:02:24.168576 17980 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebcb4fd8961f46a3a3130026fcc631be. Latest consensus state: current_term: 3 leader_uuid: "ebcb4fd8961f46a3a3130026fcc631be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcb4fd8961f46a3a3130026fcc631be" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 41897 } } }
I20260620 23:02:24.168965 17980 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [sys.catalog]: This master's current role is: LEADER
I20260620 23:02:24.168708 17979 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ebcb4fd8961f46a3a3130026fcc631be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebcb4fd8961f46a3a3130026fcc631be" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 41897 } } }
I20260620 23:02:24.169118 17979 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebcb4fd8961f46a3a3130026fcc631be [sys.catalog]: This master's current role is: LEADER
I20260620 23:02:24.180315 17985 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:02:24.189844 17985 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:02:24.191565 17985 catalog_manager.cc:1259] Loaded cluster ID: 40332be05fe444e38fdd81d798e1fba8
I20260620 23:02:24.191771 17985 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260620 23:02:24.193984 17985 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f83ba22c9ad google::LogMessage::Fail() at ??:0
@ 0x7f83ba22f88a google::LogMessage::SendToLog() at ??:0
@ 0x7f83ba22c4de google::LogMessage::Flush() at ??:0
@ 0x7f83ba22da19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f83cfc5f512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f83cfc5c976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f83cfce4610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f83cfce4439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f83bce78345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f83bce7b91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f83bce7b799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f83bce56eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f83c01306db start_thread at ??:0
@ 0x7f83b90c661f clone at ??:0
W20260620 23:02:24.498524 17976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:02:24.498966 17976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:02:24.499147 17976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:02:24.506870 17976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:02:24.507035 17976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:02:24.507220 17976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 23:02:24.517835 17976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:39699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:39915
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=46483
--tserver_master_addrs=127.2.237.62:41897
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:02:24.519378 17976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:02:24.520768 17976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:02:24.534776 18016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:24.534819 18018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:24.534917 18015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:02:24.535565 17976 server_base.cc:1034] running on GCE node
I20260620 23:02:24.536247 17976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:02:24.537354 17976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:02:24.538600 17976 hybrid_clock.cc:648] HybridClock initialized: now 1781996544538521 us; error 35 us; skew 500 ppm
I20260620 23:02:24.538915 17976 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:02:24 dist-test-slave-f63b krb5kdc[16825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996544, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:02:24.542167 17976 init.cc:377] Logged in from keytab as kudu/127.2.237.1@KRBTEST.COM (short username kudu)
I20260620 23:02:24.543810 17976 webserver.cc:460] Webserver started at http://127.2.237.1:46483/ using document root <none> and password file <none>
I20260620 23:02:24.544533 17976 fs_manager.cc:359] Metadata directory not provided
I20260620 23:02:24.544634 17976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:02:24.550488 17976 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260620 23:02:24.553678 18025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:24.555199 17976 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 23:02:24.555380 17976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "5e301f6cfd00456ebf43f313450dd98f"
format_stamp: "Formatted at 2026-06-20 23:02:18 on dist-test-slave-f63b"
server_key: "ecfcb230876b7c74e38815edb34821c6"
server_key_iv: "c2c8b6161538c8ad32e6fd89fde9925a"
server_key_version: "encryptionkey@0"
I20260620 23:02:24.556424 17976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:02:24.582736 17976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:02:24.587080 17976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:02:24.587399 17976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:02:24.588462 17976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:02:24.590368 18032 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260620 23:02:24.596410 17976 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260620 23:02:24.596545 17976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260620 23:02:24.596719 17976 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260620 23:02:24.600201 17976 ts_tablet_manager.cc:610] Registered 1 tablets
I20260620 23:02:24.600301 17976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.000s sys 0.002s
I20260620 23:02:24.600842 18032 tablet_bootstrap.cc:492] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f: Bootstrap starting.
I20260620 23:02:24.611182 18032 log.cc:824] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f: Log is configured to *not* fsync() on all Append() calls
I20260620 23:02:24.634405 18032 tablet_bootstrap.cc:492] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 23:02:24.635596 18032 tablet_bootstrap.cc:492] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f: Bootstrap complete.
I20260620 23:02:24.636359 18032 ts_tablet_manager.cc:1397] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f: Time spent bootstrapping tablet: real 0.036s user 0.027s sys 0.004s
I20260620 23:02:24.651538 18032 raft_consensus.cc:348] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44217 } } peers { permanent_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46845 } } peers { permanent_uuid: "5e301f6cfd00456ebf43f313450dd98f" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 39915 } }
I20260620 23:02:24.653771 18032 raft_consensus.cc:729] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e301f6cfd00456ebf43f313450dd98f, State: Initialized, Role: FOLLOWER
I20260620 23:02:24.654824 18032 consensus_queue.cc:260] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44217 } } peers { permanent_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46845 } } peers { permanent_uuid: "5e301f6cfd00456ebf43f313450dd98f" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 39915 } }
I20260620 23:02:24.655921 17976 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:39915
I20260620 23:02:24.655988 18140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:39915 every 8 connection(s)
I20260620 23:02:24.657148 18032 ts_tablet_manager.cc:1428] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f: Time spent starting tablet: real 0.021s user 0.017s sys 0.004s
I20260620 23:02:24.657586 17976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 23:02:24.659075 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 17976
I20260620 23:02:24.659721 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:46845
--local_ip_for_outbound_sockets=127.2.237.2
--tserver_master_addrs=127.2.237.62:41897
--webserver_port=44703
--webserver_interface=127.2.237.2
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:39699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:02:24.663801 18143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:24.661037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.1:37009)
0620 23:02:24.662178 (+ 1141us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:24.662546 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":395,"threads_started":1}
W20260620 23:02:24.664585 18141 heartbeater.cc:644] Failed to heartbeat to 127.2.237.62:41897 (0 consecutive failures): Network error: Failed to ping master at 127.2.237.62:41897: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
I20260620 23:02:24.665621 18143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:24.665242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.1:38017)
0620 23:02:24.665424 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:24.665511 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260620 23:02:24.666778 18143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:24.666351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.1:40035)
0620 23:02:24.666508 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:24.666648 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260620 23:02:24.667081 18141 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 23:02:25.004228 18144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:02:25.004557 18144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:02:25.004772 18144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:02:25.013669 18144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:02:25.013847 18144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:02:25.014062 18144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 23:02:25.024499 18144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:39699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:46845
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=44703
--tserver_master_addrs=127.2.237.62:41897
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:02:25.025928 18144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:02:25.027351 18144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:02:25.041143 18152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:25.041110 18150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:25.041234 18149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:02:25.041808 18144 server_base.cc:1034] running on GCE node
I20260620 23:02:25.042652 18144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:02:25.043654 18144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:02:25.044857 18144 hybrid_clock.cc:648] HybridClock initialized: now 1781996545044789 us; error 63 us; skew 500 ppm
I20260620 23:02:25.045215 18144 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:02:25 dist-test-slave-f63b krb5kdc[16825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996545, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:02:25.048564 18144 init.cc:377] Logged in from keytab as kudu/127.2.237.2@KRBTEST.COM (short username kudu)
I20260620 23:02:25.050181 18144 webserver.cc:460] Webserver started at http://127.2.237.2:44703/ using document root <none> and password file <none>
I20260620 23:02:25.050988 18144 fs_manager.cc:359] Metadata directory not provided
I20260620 23:02:25.051093 18144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:02:25.056970 18144 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260620 23:02:25.060102 18159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:25.061828 18144 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 23:02:25.062018 18144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "4dc7a8b876e149bd9f773b7cac8a0c30"
format_stamp: "Formatted at 2026-06-20 23:02:19 on dist-test-slave-f63b"
server_key: "6f5759a55668bbbde0588f820884d01b"
server_key_iv: "86f08585b37800801857e2f3debf34d5"
server_key_version: "encryptionkey@0"
I20260620 23:02:25.063222 18144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:02:25.085251 18144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:02:25.090025 18144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:02:25.090340 18144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:02:25.091447 18144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:02:25.092859 18144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:02:25.092948 18144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:25.093091 18144 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:02:25.093215 18144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:25.129854 18144 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:46845
I20260620 23:02:25.129879 18271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:46845 every 8 connection(s)
I20260620 23:02:25.131583 18144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 23:02:25.136895 18274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:25.134816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.2:58551)
0620 23:02:25.135629 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:25.135873 (+ 244us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":283,"threads_started":1}
I20260620 23:02:25.137099 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 18144
W20260620 23:02:25.137678 18272 heartbeater.cc:644] Failed to heartbeat to 127.2.237.62:41897 (0 consecutive failures): Network error: Failed to ping master at 127.2.237.62:41897: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
I20260620 23:02:25.137883 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:44217
--local_ip_for_outbound_sockets=127.2.237.3
--tserver_master_addrs=127.2.237.62:41897
--webserver_port=36581
--webserver_interface=127.2.237.3
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:39699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:02:25.138827 18274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:25.138417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.2:47423)
0620 23:02:25.138635 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:25.138716 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260620 23:02:25.139746 18274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:25.139397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.2:41459)
0620 23:02:25.139543 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:25.139629 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260620 23:02:25.139986 18272 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 23:02:25.487741 18275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:02:25.488082 18275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:02:25.488183 18275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:02:25.495731 18275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:02:25.495857 18275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:02:25.495985 18275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 23:02:25.506232 18275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:39699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:44217
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=36581
--tserver_master_addrs=127.2.237.62:41897
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:02:25.507547 18275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:02:25.508846 18275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:02:25.522295 18281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:25.522267 18283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:25.522341 18280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:02:25.528227 18275 server_base.cc:1034] running on GCE node
I20260620 23:02:25.528872 18275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:02:25.529831 18275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:02:25.531080 18275 hybrid_clock.cc:648] HybridClock initialized: now 1781996545531030 us; error 48 us; skew 500 ppm
I20260620 23:02:25.531411 18275 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:02:25 dist-test-slave-f63b krb5kdc[16825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996545, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:02:25.534786 18275 init.cc:377] Logged in from keytab as kudu/127.2.237.3@KRBTEST.COM (short username kudu)
I20260620 23:02:25.536408 18275 webserver.cc:460] Webserver started at http://127.2.237.3:36581/ using document root <none> and password file <none>
I20260620 23:02:25.537254 18275 fs_manager.cc:359] Metadata directory not provided
I20260620 23:02:25.537407 18275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:02:25.543466 18275 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260620 23:02:25.546545 18290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:25.548236 18275 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260620 23:02:25.548431 18275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46"
format_stamp: "Formatted at 2026-06-20 23:02:19 on dist-test-slave-f63b"
server_key: "3c4f4f339d2cfd06e390ac65416f523f"
server_key_iv: "fc7c874cb24c79f50893da20c674d697"
server_key_version: "encryptionkey@0"
I20260620 23:02:25.549499 18275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:02:25.562851 18275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:02:25.567445 18275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:02:25.567720 18275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:02:25.568753 18275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:02:25.570101 18275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:02:25.570243 18275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:25.570365 18275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:02:25.570458 18275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:25.604519 18275 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:44217
I20260620 23:02:25.604609 18402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:44217 every 8 connection(s)
I20260620 23:02:25.606061 18275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 23:02:25.611027 18405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:25.609055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.3:60987)
0620 23:02:25.609788 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:25.610032 (+ 244us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":296,"threads_started":1}
W20260620 23:02:25.611742 18403 heartbeater.cc:644] Failed to heartbeat to 127.2.237.62:41897 (0 consecutive failures): Network error: Failed to ping master at 127.2.237.62:41897: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
I20260620 23:02:25.612540 18405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:25.612220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.3:48683)
0620 23:02:25.612372 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:25.612441 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260620 23:02:25.613291 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 18275
I20260620 23:02:25.613461 18405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:25.613107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.3:39221)
0620 23:02:25.613238 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:25.613309 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260620 23:02:25.613714 18403 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 23:02:25.668503 18407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:25.667739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.1:43141)
0620 23:02:25.668204 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:25.668302 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":217,"threads_started":1}
I20260620 23:02:25.965417 18408 raft_consensus.cc:482] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:02:25.965915 18408 raft_consensus.cc:504] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44217 } } peers { permanent_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46845 } } peers { permanent_uuid: "5e301f6cfd00456ebf43f313450dd98f" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 39915 } }
I20260620 23:02:25.967816 18408 leader_election.cc:288] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5f09a35ebd6f4bb0bbe5799d06ac3a46 (127.2.237.3:44217), 4dc7a8b876e149bd9f773b7cac8a0c30 (127.2.237.2:46845)
Jun 20 23:02:25 dist-test-slave-f63b krb5kdc[16825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996544, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for kudu/127.2.237.3@KRBTEST.COM
Jun 20 23:02:25 dist-test-slave-f63b krb5kdc[16825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996544, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for kudu/127.2.237.2@KRBTEST.COM
I20260620 23:02:25.996524 18411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:25.968564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:45449 (local address 127.2.237.3:44217)
0620 23:02:25.970505 (+ 1941us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:25.970533 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:25.970604 (+ 71us) server_negotiation.cc:407] Connection header received
0620 23:02:25.972953 (+ 2349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:25.972976 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:25.973544 (+ 568us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:25.973969 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:25.976831 (+ 2862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:25.978840 (+ 2009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:25.981884 (+ 3044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:25.983028 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:25.983236 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:25.988909 (+ 5673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:25.988920 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:25.988939 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:25.988985 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:25.992179 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:25.993205 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:25.993216 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:25.993225 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:25.993407 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:25.993992 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:25.993995 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:25.993997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:25.994559 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:25.994817 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:25.995660 (+ 843us) server_negotiation.cc:299] Negotiation successful
0620 23:02:25.996063 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1721,"thread_start_us":1540,"threads_started":1}
I20260620 23:02:25.996515 18407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:25.968243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:46845 (local address 127.2.237.1:39837)
0620 23:02:25.968397 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:25.968437 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:25.969177 (+ 740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:25.974182 (+ 5005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:25.974195 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:25.975362 (+ 1167us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:02:25.976566 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:25.976598 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:25.979089 (+ 2491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:25.979093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:25.981693 (+ 2600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:25.981711 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:25.983329 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:25.983340 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:25.983827 (+ 487us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:25.984982 (+ 1155us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:02:25.985028 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:02:25.988721 (+ 3693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:02:25.992605 (+ 3884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:25.992609 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:25.992612 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:25.993079 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:25.993623 (+ 544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:25.993627 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:25.993629 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:25.993813 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:25.994758 (+ 945us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:02:25.994768 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:02:25.995134 (+ 366us) client_negotiation.cc:769] Sending connection context
0620 23:02:25.995430 (+ 296us) client_negotiation.cc:240] Negotiation successful
0620 23:02:25.996075 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":491,"spinlock_wait_cycles":38656}
I20260620 23:02:25.996546 18410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:25.968474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:39837 (local address 127.2.237.2:46845)
0620 23:02:25.970699 (+ 2225us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:25.970712 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:25.970743 (+ 31us) server_negotiation.cc:407] Connection header received
0620 23:02:25.972953 (+ 2210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:25.972967 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:25.973551 (+ 584us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:25.973956 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:25.976831 (+ 2875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:25.978840 (+ 2009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:25.981883 (+ 3043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:25.983028 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:25.983236 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:25.989038 (+ 5802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:25.989047 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:25.989060 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:25.989108 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:25.992434 (+ 3326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:25.993266 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:25.993274 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:25.993279 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:25.993487 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:25.993954 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:25.993962 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:25.993966 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:25.994542 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:25.994817 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:25.995657 (+ 840us) server_negotiation.cc:299] Negotiation successful
0620 23:02:25.996045 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2039,"thread_start_us":1904,"threads_started":1}
I20260620 23:02:25.996790 18409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:25.968249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:44217 (local address 127.2.237.1:45449)
0620 23:02:25.970067 (+ 1818us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:25.970087 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:25.970207 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:25.974182 (+ 3975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:25.974195 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:25.975372 (+ 1177us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:02:25.976566 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:25.976598 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:25.979087 (+ 2489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:25.979093 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:25.981693 (+ 2600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:25.981711 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:25.983445 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:25.983453 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:25.983824 (+ 371us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:25.984982 (+ 1158us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:02:25.985028 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:02:25.988510 (+ 3482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:02:25.992398 (+ 3888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:25.992409 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:25.992426 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:25.993046 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:25.993553 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:25.993561 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:25.993566 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:25.993830 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:25.994778 (+ 948us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:02:25.994784 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:02:25.995134 (+ 350us) client_negotiation.cc:769] Sending connection context
0620 23:02:25.995428 (+ 294us) client_negotiation.cc:240] Negotiation successful
0620 23:02:25.996283 (+ 855us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1678,"mutex_wait_us":876,"spinlock_wait_cycles":40704,"thread_start_us":1298,"threads_started":1}
W20260620 23:02:26.000497 18027 leader_election.cc:341] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5f09a35ebd6f4bb0bbe5799d06ac3a46 (127.2.237.3:44217): Not found: Tablet not found: 5b7b9d2c46514007a68a2bf50d14558b
W20260620 23:02:26.000797 18028 leader_election.cc:341] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4dc7a8b876e149bd9f773b7cac8a0c30 (127.2.237.2:46845): Not found: Tablet not found: 5b7b9d2c46514007a68a2bf50d14558b
I20260620 23:02:26.000985 18028 leader_election.cc:302] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e301f6cfd00456ebf43f313450dd98f; no voters: 4dc7a8b876e149bd9f773b7cac8a0c30, 5f09a35ebd6f4bb0bbe5799d06ac3a46
I20260620 23:02:26.001477 18408 raft_consensus.cc:2726] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:02:26.141577 18412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:26.140577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.2:50147)
0620 23:02:26.141107 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:26.141320 (+ 213us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":247,"threads_started":1}
I20260620 23:02:26.615106 18413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:26.614305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.3:38499)
0620 23:02:26.614803 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:26.614905 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":289,"threads_started":1}
I20260620 23:02:26.670344 18414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:26.669532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.1:32939)
0620 23:02:26.670062 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:26.670140 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":292,"threads_started":1}
I20260620 23:02:27.143348 18415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:27.142568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.2:35139)
0620 23:02:27.143082 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:27.143170 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":225,"threads_started":1}
I20260620 23:02:27.616933 18416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:27.616173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.3:34581)
0620 23:02:27.616638 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:27.616730 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":228,"threads_started":1}
I20260620 23:02:27.672343 18417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:27.671481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.1:33631)
0620 23:02:27.672057 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:27.672147 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":246,"threads_started":1}
I20260620 23:02:28.129853 18418 raft_consensus.cc:482] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:02:28.130085 18418 raft_consensus.cc:504] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44217 } } peers { permanent_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46845 } } peers { permanent_uuid: "5e301f6cfd00456ebf43f313450dd98f" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 39915 } }
I20260620 23:02:28.130889 18418 leader_election.cc:288] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5f09a35ebd6f4bb0bbe5799d06ac3a46 (127.2.237.3:44217), 4dc7a8b876e149bd9f773b7cac8a0c30 (127.2.237.2:46845)
W20260620 23:02:28.131909 18027 leader_election.cc:341] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5f09a35ebd6f4bb0bbe5799d06ac3a46 (127.2.237.3:44217): Not found: Tablet not found: 5b7b9d2c46514007a68a2bf50d14558b
W20260620 23:02:28.132097 18028 leader_election.cc:341] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4dc7a8b876e149bd9f773b7cac8a0c30 (127.2.237.2:46845): Not found: Tablet not found: 5b7b9d2c46514007a68a2bf50d14558b
I20260620 23:02:28.132256 18028 leader_election.cc:302] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e301f6cfd00456ebf43f313450dd98f; no voters: 4dc7a8b876e149bd9f773b7cac8a0c30, 5f09a35ebd6f4bb0bbe5799d06ac3a46
I20260620 23:02:28.132541 18418 raft_consensus.cc:2726] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:02:28.145186 18419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:28.144423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.2:46931)
0620 23:02:28.144929 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:28.145018 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":252,"threads_started":1}
I20260620 23:02:28.618768 18420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:28.617937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.3:52541)
0620 23:02:28.618442 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:28.618547 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":301,"threads_started":1}
I20260620 23:02:28.674293 18421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:28.673470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.1:60461)
0620 23:02:28.674023 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:28.674116 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":277,"threads_started":1}
I20260620 23:02:29.147035 18422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:29.146203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.2:51839)
0620 23:02:29.146749 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:29.146839 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":231,"threads_started":1}
I20260620 23:02:29.620596 18423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:29.619770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.3:57133)
0620 23:02:29.620266 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:29.620369 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":236,"threads_started":1}
I20260620 23:02:29.676200 18424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:29.675396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.1:59969)
0620 23:02:29.675917 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:29.676020 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":241,"threads_started":1}
I20260620 23:02:30.148653 18425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:30.147974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.2:35181)
0620 23:02:30.148404 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:30.148483 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":225,"threads_started":1}
I20260620 23:02:30.622629 18426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:30.621719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.3:58405)
0620 23:02:30.622269 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:30.622363 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":320,"threads_started":1}
I20260620 23:02:30.678058 18427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:30.677267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.1:41669)
0620 23:02:30.677772 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:30.677868 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":254,"threads_started":1}
I20260620 23:02:31.150419 18428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:31.149666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.2:50537)
0620 23:02:31.150141 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:31.150228 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":242,"threads_started":1}
I20260620 23:02:31.427984 18429 raft_consensus.cc:482] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:02:31.428205 18429 raft_consensus.cc:504] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44217 } } peers { permanent_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46845 } } peers { permanent_uuid: "5e301f6cfd00456ebf43f313450dd98f" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 39915 } }
I20260620 23:02:31.428817 18429 leader_election.cc:288] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5f09a35ebd6f4bb0bbe5799d06ac3a46 (127.2.237.3:44217), 4dc7a8b876e149bd9f773b7cac8a0c30 (127.2.237.2:46845)
W20260620 23:02:31.429970 18028 leader_election.cc:341] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4dc7a8b876e149bd9f773b7cac8a0c30 (127.2.237.2:46845): Not found: Tablet not found: 5b7b9d2c46514007a68a2bf50d14558b
W20260620 23:02:31.430181 18027 leader_election.cc:341] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5f09a35ebd6f4bb0bbe5799d06ac3a46 (127.2.237.3:44217): Not found: Tablet not found: 5b7b9d2c46514007a68a2bf50d14558b
I20260620 23:02:31.430307 18027 leader_election.cc:302] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e301f6cfd00456ebf43f313450dd98f; no voters: 4dc7a8b876e149bd9f773b7cac8a0c30, 5f09a35ebd6f4bb0bbe5799d06ac3a46
I20260620 23:02:31.430642 18429 raft_consensus.cc:2726] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:02:31.624374 18430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:31.623640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.3:45831)
0620 23:02:31.624113 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:31.624192 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":245,"threads_started":1}
I20260620 23:02:31.680210 18431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:31.679206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.1:54107)
0620 23:02:31.679836 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:31.679939 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":369,"threads_started":1}
I20260620 23:02:32.152398 18432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:32.151526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.2:37933)
0620 23:02:32.152072 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:32.152167 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":230,"threads_started":1}
I20260620 23:02:32.626286 18433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:32.625444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.3:54557)
0620 23:02:32.625968 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:32.626053 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":256,"threads_started":1}
I20260620 23:02:32.681902 18434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:32.681181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.1:45961)
0620 23:02:32.681654 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:32.681737 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":244,"threads_started":1}
I20260620 23:02:33.154300 18435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:33.153421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.2:53501)
0620 23:02:33.153972 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:33.154063 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":222,"threads_started":1}
I20260620 23:02:33.628129 18436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:33.627408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.3:42355)
0620 23:02:33.627857 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:33.627945 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":236,"threads_started":1}
I20260620 23:02:33.683691 18437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:33.682912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.1:36205)
0620 23:02:33.683386 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:33.683482 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":237,"threads_started":1}
I20260620 23:02:33.807324 18438 raft_consensus.cc:482] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:02:33.807659 18438 raft_consensus.cc:504] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f09a35ebd6f4bb0bbe5799d06ac3a46" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44217 } } peers { permanent_uuid: "4dc7a8b876e149bd9f773b7cac8a0c30" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46845 } } peers { permanent_uuid: "5e301f6cfd00456ebf43f313450dd98f" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 39915 } }
I20260620 23:02:33.808429 18438 leader_election.cc:288] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5f09a35ebd6f4bb0bbe5799d06ac3a46 (127.2.237.3:44217), 4dc7a8b876e149bd9f773b7cac8a0c30 (127.2.237.2:46845)
W20260620 23:02:33.809356 18027 leader_election.cc:341] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5f09a35ebd6f4bb0bbe5799d06ac3a46 (127.2.237.3:44217): Not found: Tablet not found: 5b7b9d2c46514007a68a2bf50d14558b
W20260620 23:02:33.809552 18028 leader_election.cc:341] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4dc7a8b876e149bd9f773b7cac8a0c30 (127.2.237.2:46845): Not found: Tablet not found: 5b7b9d2c46514007a68a2bf50d14558b
I20260620 23:02:33.809717 18028 leader_election.cc:302] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e301f6cfd00456ebf43f313450dd98f; no voters: 4dc7a8b876e149bd9f773b7cac8a0c30, 5f09a35ebd6f4bb0bbe5799d06ac3a46
I20260620 23:02:33.809989 18438 raft_consensus.cc:2726] T 5b7b9d2c46514007a68a2bf50d14558b P 5e301f6cfd00456ebf43f313450dd98f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260620 23:02:34.156418 18439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:34.155504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.2:53939)
0620 23:02:34.156108 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:34.156212 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":326,"threads_started":1}
I20260620 23:02:34.630144 18440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:34.629297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.3:56227)
0620 23:02:34.629854 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:34.629957 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":276,"threads_started":1}
I20260620 23:02:34.685555 18441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:34.684771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.1:47815)
0620 23:02:34.685274 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:34.685371 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":241,"threads_started":1}
I20260620 23:02:35.158246 18442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:35.157487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.2:55807)
0620 23:02:35.157948 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:35.158061 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":240,"threads_started":1}
I20260620 23:02:35.631908 18443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:35.631149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.3:38391)
0620 23:02:35.631658 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:35.631737 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":275,"threads_started":1}
I20260620 23:02:35.687482 18444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:35.686732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.1:56239)
0620 23:02:35.687196 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:35.687287 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":249,"threads_started":1}
I20260620 23:02:36.159960 18445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:36.159188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.2:43127)
0620 23:02:36.159688 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:36.159776 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":248,"threads_started":1}
I20260620 23:02:36.634100 18446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:36.633025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.3:54403)
0620 23:02:36.633845 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:36.633934 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":363,"threads_started":1}
I20260620 23:02:36.689298 18447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:36.688497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.1:44417)
0620 23:02:36.689012 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:36.689124 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":276,"threads_started":1}
I20260620 23:02:37.161818 18448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:37.160934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.2:42685)
0620 23:02:37.161506 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:37.161626 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":346,"threads_started":1}
I20260620 23:02:37.635887 18449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:37.635140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.3:47773)
0620 23:02:37.635612 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:37.635708 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":238,"threads_started":1}
I20260620 23:02:37.691438 18450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:37.690461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.1:54839)
0620 23:02:37.691148 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:37.691256 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":430,"threads_started":1}
I20260620 23:02:38.163764 18451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:38.162943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.2:52671)
0620 23:02:38.163464 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:38.163583 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":248,"threads_started":1}
I20260620 23:02:38.637835 18452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:38.637032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.3:46535)
0620 23:02:38.637533 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:38.637636 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":257,"threads_started":1}
I20260620 23:02:38.693361 18453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:38.692461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.1:37627)
0620 23:02:38.693045 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:38.693171 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":326,"threads_started":1}
I20260620 23:02:39.165539 18454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:39.164741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.2:39437)
0620 23:02:39.165246 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:39.165337 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":258,"threads_started":1}
I20260620 23:02:39.639822 18455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:39.639037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.3:39777)
0620 23:02:39.639532 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:39.639619 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":255,"threads_started":1}
I20260620 23:02:39.695533 18456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:39.694769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.1:59999)
0620 23:02:39.695247 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:39.695344 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":222,"threads_started":1}
I20260620 23:02:40.167586 18457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:40.166871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.2:37079)
0620 23:02:40.167317 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:40.167428 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":218,"threads_started":1}
I20260620 23:02:40.617066 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 17976
I20260620 23:02:40.636999 18458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:40.618436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46750 (local address 127.2.237.1:39915)
0620 23:02:40.619064 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:40.619072 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:40.619100 (+ 28us) server_negotiation.cc:407] Connection header received
0620 23:02:40.619165 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:40.619172 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:40.619434 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:40.619616 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:40.621306 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:40.622312 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:40.623749 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:40.624357 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:40.624521 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:40.630855 (+ 6334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:40.630873 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:40.630890 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:40.630932 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:40.633845 (+ 2913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:40.634674 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:40.634683 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:40.634685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:40.634835 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:40.635259 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:40.635263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:40.635265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:40.635661 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:40.636023 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:40.636347 (+ 324us) server_negotiation.cc:299] Negotiation successful
0620 23:02:40.636649 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":264,"threads_started":1}
I20260620 23:02:40.638391 18135 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:02:40.641965 18460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:40.641142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:41897 (local address 127.2.237.3:33557)
0620 23:02:40.641630 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:40.641758 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:41897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":235,"threads_started":1}
I20260620 23:02:40.735965 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 17976
I20260620 23:02:40.755407 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 18144
I20260620 23:02:40.769814 18462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:40.756149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37380 (local address 127.2.237.2:46845)
0620 23:02:40.756655 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:40.756663 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:40.756687 (+ 24us) server_negotiation.cc:407] Connection header received
0620 23:02:40.756760 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:40.756764 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:40.756828 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:40.756925 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:40.758294 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:40.759252 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:40.760524 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:40.760981 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:40.761100 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:40.763838 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:40.763844 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:40.763848 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:40.763894 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:40.766949 (+ 3055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:40.767812 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:40.767816 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:40.767819 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:40.767977 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:40.768398 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:40.768401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:40.768403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:40.768691 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:40.768906 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:40.769241 (+ 335us) server_negotiation.cc:299] Negotiation successful
0620 23:02:40.769466 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":269,"threads_started":1}
I20260620 23:02:40.770500 18267 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:02:40.857002 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 18144
I20260620 23:02:40.874817 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 18275
I20260620 23:02:40.893971 18464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:40.875631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58284 (local address 127.2.237.3:44217)
0620 23:02:40.876296 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:40.876306 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:40.876332 (+ 26us) server_negotiation.cc:407] Connection header received
0620 23:02:40.876404 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:40.876410 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:40.876511 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:40.876635 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:40.878716 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:40.879816 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:40.881152 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:40.881604 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:40.881778 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:40.886312 (+ 4534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:40.886320 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:40.886328 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:40.886388 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:40.890416 (+ 4028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:40.891734 (+ 1318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:40.891740 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:40.891744 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:40.891916 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:40.892434 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:40.892440 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:40.892443 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:40.892737 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:40.893042 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:40.893323 (+ 281us) server_negotiation.cc:299] Negotiation successful
0620 23:02:40.893562 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":1238,"threads_started":1}
I20260620 23:02:40.894877 18398 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:02:40.994021 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 18275
2026-06-20T23:02:41Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24623 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:02:41 dist-test-slave-f63b krb5kdc[18468](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:02:41 dist-test-slave-f63b krb5kdc[18468](info): set up 2 sockets
Jun 20 23:02:41 dist-test-slave-f63b krb5kdc[18468](info): commencing operation
krb5kdc: starting...
W20260620 23:02:42.103309 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.005s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 23:02:42 dist-test-slave-f63b krb5kdc[18468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:02:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:02:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
I20260620 23:02:42.354835 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:43887
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:43543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:43887
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:02:42.714087 18481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:02:42.714470 18481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:02:42.714664 18481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:02:42.722312 18481 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260620 23:02:42.722493 18481 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260620 23:02:42.722563 18481 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260620 23:02:42.722669 18481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:02:42.722747 18481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:02:42.722851 18481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:02:42.722944 18481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:02:42.723028 18481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:02:42.733650 18481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:43543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:43887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:43887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:02:42.735220 18481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:02:42.736774 18481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:02:42.749621 18487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:42.749681 18486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:42.750396 18489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:02:42.750413 18481 server_base.cc:1034] running on GCE node
I20260620 23:02:42.751478 18481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:02:42.753106 18481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:02:42.754390 18481 hybrid_clock.cc:648] HybridClock initialized: now 1781996562754309 us; error 57 us; skew 500 ppm
I20260620 23:02:42.754837 18481 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:02:42 dist-test-slave-f63b krb5kdc[18468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996562, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:02:42.758261 18481 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 23:02:42.759881 18481 webserver.cc:460] Webserver started at http://127.2.237.62:36741/ using document root <none> and password file <none>
I20260620 23:02:42.760591 18481 fs_manager.cc:359] Metadata directory not provided
I20260620 23:02:42.760680 18481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:02:42.760947 18481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:02:42.764402 18481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "97fc1aadd98c4012b792727ecb1eaf05"
format_stamp: "Formatted at 2026-06-20 23:02:42 on dist-test-slave-f63b"
server_key: "30793aea8b7b5f74a90d93a6700aca15"
server_key_iv: "6c53b05c23f3faf75ef8c113ce2fa582"
server_key_version: "encryptionkey@0"
I20260620 23:02:42.765219 18481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "97fc1aadd98c4012b792727ecb1eaf05"
format_stamp: "Formatted at 2026-06-20 23:02:42 on dist-test-slave-f63b"
server_key: "30793aea8b7b5f74a90d93a6700aca15"
server_key_iv: "6c53b05c23f3faf75ef8c113ce2fa582"
server_key_version: "encryptionkey@0"
I20260620 23:02:42.771142 18481 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260620 23:02:42.775065 18496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:42.776654 18481 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260620 23:02:42.776813 18481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "97fc1aadd98c4012b792727ecb1eaf05"
format_stamp: "Formatted at 2026-06-20 23:02:42 on dist-test-slave-f63b"
server_key: "30793aea8b7b5f74a90d93a6700aca15"
server_key_iv: "6c53b05c23f3faf75ef8c113ce2fa582"
server_key_version: "encryptionkey@0"
I20260620 23:02:42.777015 18481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:02:42.797245 18481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:02:42.802206 18481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:02:42.802440 18481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:02:42.821811 18481 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:43887
I20260620 23:02:42.821832 18547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:43887 every 8 connection(s)
I20260620 23:02:42.823398 18481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:02:42.827716 18548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:02:42.830832 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 18481
I20260620 23:02:42.831053 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:02:42.831772 2996 external_mini_cluster.cc:1428] Setting key 1a5310c0a151755e8327b98c5a20e03f
I20260620 23:02:42.840478 18548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05: Bootstrap starting.
I20260620 23:02:42.844002 18548 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05: Neither blocks nor log segments found. Creating new log.
I20260620 23:02:42.845134 18548 log.cc:824] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05: Log is configured to *not* fsync() on all Append() calls
Jun 20 23:02:42 dist-test-slave-f63b krb5kdc[18468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:02:42.848527 18548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05: No bootstrap required, opened a new log
I20260620 23:02:42.856243 18551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:42.834121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55894 (local address 127.2.237.62:43887)
0620 23:02:42.835013 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:42.835026 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:42.835071 (+ 45us) server_negotiation.cc:407] Connection header received
0620 23:02:42.836869 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:42.836894 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:42.837358 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:42.837820 (+ 462us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:42.839459 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:42.840841 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:42.842316 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:42.843231 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:42.843445 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:42.848260 (+ 4815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:42.848268 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:42.848302 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:42.848342 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:42.851769 (+ 3427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:42.852584 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:42.852591 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:42.852598 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:42.852781 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:42.853219 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:42.853229 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:42.853234 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:42.853834 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:42.854188 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:42.854655 (+ 467us) server_negotiation.cc:299] Negotiation successful
0620 23:02:42.855141 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":426,"threads_started":1}
I20260620 23:02:42.856882 18548 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97fc1aadd98c4012b792727ecb1eaf05" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43887 } }
I20260620 23:02:42.857383 18548 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:02:42.857498 18548 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97fc1aadd98c4012b792727ecb1eaf05, State: Initialized, Role: FOLLOWER
I20260620 23:02:42.858115 18548 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97fc1aadd98c4012b792727ecb1eaf05" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43887 } }
I20260620 23:02:42.858387 18548 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:02:42.858566 18548 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:02:42.858811 18548 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:02:42.861378 18548 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97fc1aadd98c4012b792727ecb1eaf05" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43887 } }
I20260620 23:02:42.861955 18548 leader_election.cc:302] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97fc1aadd98c4012b792727ecb1eaf05; no voters:
I20260620 23:02:42.862704 18548 leader_election.cc:288] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:02:42.862875 18553 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:02:42.863860 18553 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [term 1 LEADER]: Becoming Leader. State: Replica: 97fc1aadd98c4012b792727ecb1eaf05, State: Running, Role: LEADER
I20260620 23:02:42.864527 18553 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97fc1aadd98c4012b792727ecb1eaf05" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43887 } }
I20260620 23:02:42.865305 18548 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:02:42.867928 18555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97fc1aadd98c4012b792727ecb1eaf05. Latest consensus state: current_term: 1 leader_uuid: "97fc1aadd98c4012b792727ecb1eaf05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97fc1aadd98c4012b792727ecb1eaf05" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43887 } } }
I20260620 23:02:42.869165 18554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97fc1aadd98c4012b792727ecb1eaf05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97fc1aadd98c4012b792727ecb1eaf05" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43887 } } }
I20260620 23:02:42.869572 18554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [sys.catalog]: This master's current role is: LEADER
I20260620 23:02:42.869580 18555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [sys.catalog]: This master's current role is: LEADER
I20260620 23:02:42.870807 18562 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:02:42.876683 18562 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:02:42.887313 18562 catalog_manager.cc:1347] Generated new cluster ID: 5a37db2db7124067851f01dfc1227a10
I20260620 23:02:42.887472 18562 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:02:42.897071 18562 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:02:42.898487 18562 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:02:42.910426 18562 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05: Generated new TSK 0
I20260620 23:02:42.911455 18562 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.237.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
I20260620 23:02:42.979423 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:0
--local_ip_for_outbound_sockets=127.2.237.1
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:43887
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:43543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:02:43.335407 18574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:02:43.335724 18574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:02:43.335909 18574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:02:43.343530 18574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:02:43.343685 18574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:02:43.343881 18574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 23:02:43.354651 18574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:43543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:43887
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:02:43.356047 18574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:02:43.357441 18574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:02:43.371587 18580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:43.371665 18579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:43.371642 18582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:02:43.377550 18574 server_base.cc:1034] running on GCE node
I20260620 23:02:43.378305 18574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:02:43.379489 18574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:02:43.380775 18574 hybrid_clock.cc:648] HybridClock initialized: now 1781996563380678 us; error 61 us; skew 500 ppm
I20260620 23:02:43.381148 18574 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:02:43 dist-test-slave-f63b krb5kdc[18468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996563, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:02:43.384635 18574 init.cc:377] Logged in from keytab as kudu/127.2.237.1@KRBTEST.COM (short username kudu)
I20260620 23:02:43.386343 18574 webserver.cc:460] Webserver started at http://127.2.237.1:34071/ using document root <none> and password file <none>
I20260620 23:02:43.387176 18574 fs_manager.cc:359] Metadata directory not provided
I20260620 23:02:43.387315 18574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:02:43.387663 18574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:02:43.390873 18574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/data/instance:
uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5"
format_stamp: "Formatted at 2026-06-20 23:02:43 on dist-test-slave-f63b"
server_key: "32a3d3fc08a9f357ad3d7a77590da64f"
server_key_iv: "a141a099226d5fa855cd08f6d93122ba"
server_key_version: "encryptionkey@0"
I20260620 23:02:43.391669 18574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance:
uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5"
format_stamp: "Formatted at 2026-06-20 23:02:43 on dist-test-slave-f63b"
server_key: "32a3d3fc08a9f357ad3d7a77590da64f"
server_key_iv: "a141a099226d5fa855cd08f6d93122ba"
server_key_version: "encryptionkey@0"
I20260620 23:02:43.397240 18574 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260620 23:02:43.400992 18589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:43.402485 18574 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 23:02:43.402699 18574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5"
format_stamp: "Formatted at 2026-06-20 23:02:43 on dist-test-slave-f63b"
server_key: "32a3d3fc08a9f357ad3d7a77590da64f"
server_key_iv: "a141a099226d5fa855cd08f6d93122ba"
server_key_version: "encryptionkey@0"
I20260620 23:02:43.402938 18574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:02:43.438880 18574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:02:43.443408 18574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:02:43.443746 18574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:02:43.444880 18574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:02:43.446355 18574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:02:43.446468 18574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:43.446650 18574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:02:43.446733 18574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:43.482259 18574 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:41657
I20260620 23:02:43.482321 18701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:41657 every 8 connection(s)
I20260620 23:02:43.483733 18574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 23:02:43.488161 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 18574
I20260620 23:02:43.488440 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance
I20260620 23:02:43.489312 2996 external_mini_cluster.cc:1428] Setting key 1889f9d62283d97d8717505d73278c65
Jun 20 23:02:43 dist-test-slave-f63b krb5kdc[18468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996563, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:02:43.509277 18704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:43.486641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:50315 (local address 127.2.237.62:43887)
0620 23:02:43.487092 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:43.487097 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:43.488436 (+ 1339us) server_negotiation.cc:407] Connection header received
0620 23:02:43.490914 (+ 2478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:43.490918 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:43.490986 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:43.491085 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:43.493291 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:43.494014 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:43.496589 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:43.496982 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:43.497084 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:43.501610 (+ 4526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:43.501619 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:43.501627 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:43.501675 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:43.505447 (+ 3772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:43.506401 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:43.506409 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:43.506416 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:43.506607 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:43.507091 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:43.507099 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:43.507105 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:43.507513 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:43.507667 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:43.508497 (+ 830us) server_negotiation.cc:299] Negotiation successful
0620 23:02:43.508765 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":221,"threads_started":1}
I20260620 23:02:43.509938 18705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:43.487252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.1:50315)
0620 23:02:43.488121 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:43.488190 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:43.490555 (+ 2365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:43.491395 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:43.491404 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:43.492190 (+ 786us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:02:43.493075 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:43.493105 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:43.494194 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:43.494199 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:43.496431 (+ 2232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:43.496449 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:43.497241 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:43.497248 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:43.497583 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:43.498349 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:02:43.498390 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:02:43.501333 (+ 2943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:02:43.505699 (+ 4366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:43.505708 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:43.505724 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:43.506214 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:43.506727 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:43.506731 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:43.506733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:43.506963 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:43.507641 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:02:43.507647 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:02:43.508056 (+ 409us) client_negotiation.cc:769] Sending connection context
0620 23:02:43.508314 (+ 258us) client_negotiation.cc:240] Negotiation successful
0620 23:02:43.508756 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":320,"threads_started":1}
I20260620 23:02:43.511684 18702 heartbeater.cc:347] Connected to a master server at 127.2.237.62:43887
I20260620 23:02:43.512023 18702 heartbeater.cc:464] Registering TS with master...
I20260620 23:02:43.512897 18702 heartbeater.cc:511] Master 127.2.237.62:43887 requested a full tablet report, sending...
I20260620 23:02:43.515202 18513 ts_manager.cc:194] Registered new tserver with Master: 66cfdaa2f4c143feac2c5a2d76bbdac5 (127.2.237.1:41657)
WARNING: no policy specified for kudu/127.2.237.2@KRBTEST.COM; defaulting to no policy
I20260620 23:02:43.516790 18513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.1@KRBTEST.COM'} at 127.2.237.1:50315
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
I20260620 23:02:43.548573 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:0
--local_ip_for_outbound_sockets=127.2.237.2
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:43887
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:43543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:02:43.892848 18708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:02:43.893190 18708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:02:43.893368 18708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:02:43.901104 18708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:02:43.901237 18708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:02:43.901450 18708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 23:02:43.912120 18708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:43543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:43887
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:02:43.913553 18708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:02:43.915019 18708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:02:43.929067 18713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:43.929111 18716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:43.929168 18714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:02:43.930058 18708 server_base.cc:1034] running on GCE node
I20260620 23:02:43.930784 18708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:02:43.931806 18708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:02:43.933089 18708 hybrid_clock.cc:648] HybridClock initialized: now 1781996563933011 us; error 68 us; skew 500 ppm
I20260620 23:02:43.933386 18708 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:02:43 dist-test-slave-f63b krb5kdc[18468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996563, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:02:43.936748 18708 init.cc:377] Logged in from keytab as kudu/127.2.237.2@KRBTEST.COM (short username kudu)
I20260620 23:02:43.938460 18708 webserver.cc:460] Webserver started at http://127.2.237.2:46673/ using document root <none> and password file <none>
I20260620 23:02:43.939226 18708 fs_manager.cc:359] Metadata directory not provided
I20260620 23:02:43.939380 18708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:02:43.939708 18708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:02:43.942883 18708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/data/instance:
uuid: "42394c1c54b646f08ed66bc1329c070e"
format_stamp: "Formatted at 2026-06-20 23:02:43 on dist-test-slave-f63b"
server_key: "e8b86a2cce2ea4b5a7a698f6cc4b7445"
server_key_iv: "f83acd279c5d076543306ce5be9db94e"
server_key_version: "encryptionkey@0"
I20260620 23:02:43.943719 18708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance:
uuid: "42394c1c54b646f08ed66bc1329c070e"
format_stamp: "Formatted at 2026-06-20 23:02:43 on dist-test-slave-f63b"
server_key: "e8b86a2cce2ea4b5a7a698f6cc4b7445"
server_key_iv: "f83acd279c5d076543306ce5be9db94e"
server_key_version: "encryptionkey@0"
I20260620 23:02:43.949355 18708 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260620 23:02:43.953347 18723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:43.955025 18708 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260620 23:02:43.955245 18708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "42394c1c54b646f08ed66bc1329c070e"
format_stamp: "Formatted at 2026-06-20 23:02:43 on dist-test-slave-f63b"
server_key: "e8b86a2cce2ea4b5a7a698f6cc4b7445"
server_key_iv: "f83acd279c5d076543306ce5be9db94e"
server_key_version: "encryptionkey@0"
I20260620 23:02:43.955506 18708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:02:43.975835 18708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:02:43.981350 18708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:02:43.981683 18708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:02:43.982873 18708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:02:43.984385 18708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:02:43.984500 18708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:43.984658 18708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:02:43.984736 18708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:44.020974 18708 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:34287
I20260620 23:02:44.021008 18835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:34287 every 8 connection(s)
I20260620 23:02:44.022343 18708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 23:02:44.024129 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 18708
I20260620 23:02:44.024331 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance
I20260620 23:02:44.024899 2996 external_mini_cluster.cc:1428] Setting key c2924006e4048e9f8d8cb2dce6615e6f
Jun 20 23:02:44 dist-test-slave-f63b krb5kdc[18468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996563, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.237.3@KRBTEST.COM; defaulting to no policy
I20260620 23:02:44.048949 18838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:44.025167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:40027 (local address 127.2.237.62:43887)
0620 23:02:44.025657 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:44.025661 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:44.026904 (+ 1243us) server_negotiation.cc:407] Connection header received
0620 23:02:44.032840 (+ 5936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:44.032846 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:44.032957 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:44.033102 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:44.034999 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.035869 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.037578 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.038023 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.038198 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:44.042344 (+ 4146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:44.042349 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:44.042353 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:44.042392 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:44.045351 (+ 2959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:44.046287 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:44.046292 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:44.046296 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:44.046433 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:44.046939 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:44.046949 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:44.046951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:44.047241 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:44.047409 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:44.048270 (+ 861us) server_negotiation.cc:299] Negotiation successful
0620 23:02:44.048543 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":253,"threads_started":1}
I20260620 23:02:44.050043 18839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:44.025846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.2:40027)
0620 23:02:44.026727 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:44.026776 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:44.032459 (+ 5683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:44.033396 (+ 937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:44.033405 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:44.033986 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:02:44.034784 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:44.034806 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.036066 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.036070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:44.037406 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:44.037419 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.038299 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.038306 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:44.038555 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:44.039388 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:02:44.039417 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:02:44.042042 (+ 2625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:02:44.045530 (+ 3488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:44.045540 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:44.045559 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:44.046123 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:44.046559 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:44.046565 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:44.046584 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:44.046800 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:44.047382 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:02:44.047391 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:02:44.047860 (+ 469us) client_negotiation.cc:769] Sending connection context
0620 23:02:44.048177 (+ 317us) client_negotiation.cc:240] Negotiation successful
0620 23:02:44.048839 (+ 662us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":372,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.3@KRBTEST.COM" created.
I20260620 23:02:44.051688 18836 heartbeater.cc:347] Connected to a master server at 127.2.237.62:43887
I20260620 23:02:44.052021 18836 heartbeater.cc:464] Registering TS with master...
I20260620 23:02:44.052767 18836 heartbeater.cc:511] Master 127.2.237.62:43887 requested a full tablet report, sending...
I20260620 23:02:44.054382 18513 ts_manager.cc:194] Registered new tserver with Master: 42394c1c54b646f08ed66bc1329c070e (127.2.237.2:34287)
I20260620 23:02:44.055270 18513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.2@KRBTEST.COM'} at 127.2.237.2:40027
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
I20260620 23:02:44.082180 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:0
--local_ip_for_outbound_sockets=127.2.237.3
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:43887
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:43543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:02:44.425673 18842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:02:44.425968 18842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:02:44.426061 18842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:02:44.433570 18842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:02:44.433701 18842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:02:44.433815 18842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 23:02:44.443996 18842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:43543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:43887
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:02:44.445324 18842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:02:44.446647 18842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:02:44.460502 18847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:44.460433 18848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:02:44.460407 18850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:02:44.461604 18842 server_base.cc:1034] running on GCE node
I20260620 23:02:44.462291 18842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:02:44.463421 18842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:02:44.464696 18842 hybrid_clock.cc:648] HybridClock initialized: now 1781996564464659 us; error 65 us; skew 500 ppm
I20260620 23:02:44.465102 18842 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:02:44 dist-test-slave-f63b krb5kdc[18468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996564, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:02:44.468736 18842 init.cc:377] Logged in from keytab as kudu/127.2.237.3@KRBTEST.COM (short username kudu)
I20260620 23:02:44.470511 18842 webserver.cc:460] Webserver started at http://127.2.237.3:34131/ using document root <none> and password file <none>
I20260620 23:02:44.471284 18842 fs_manager.cc:359] Metadata directory not provided
I20260620 23:02:44.471437 18842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:02:44.471778 18842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:02:44.474915 18842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/data/instance:
uuid: "a2ed604e389e43338b71e4b41802334a"
format_stamp: "Formatted at 2026-06-20 23:02:44 on dist-test-slave-f63b"
server_key: "797655a11b17dcdfddde67a64dda03e5"
server_key_iv: "f11e95e0a79d403f4c8e322762e9eea2"
server_key_version: "encryptionkey@0"
I20260620 23:02:44.475749 18842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance:
uuid: "a2ed604e389e43338b71e4b41802334a"
format_stamp: "Formatted at 2026-06-20 23:02:44 on dist-test-slave-f63b"
server_key: "797655a11b17dcdfddde67a64dda03e5"
server_key_iv: "f11e95e0a79d403f4c8e322762e9eea2"
server_key_version: "encryptionkey@0"
I20260620 23:02:44.481359 18842 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260620 23:02:44.486152 18857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:44.487972 18842 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260620 23:02:44.488183 18842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "a2ed604e389e43338b71e4b41802334a"
format_stamp: "Formatted at 2026-06-20 23:02:44 on dist-test-slave-f63b"
server_key: "797655a11b17dcdfddde67a64dda03e5"
server_key_iv: "f11e95e0a79d403f4c8e322762e9eea2"
server_key_version: "encryptionkey@0"
I20260620 23:02:44.488404 18842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:02:44.509675 18842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:02:44.513896 18842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:02:44.514156 18842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:02:44.515170 18842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:02:44.516569 18842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:02:44.516649 18842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:44.516734 18842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:02:44.516778 18842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:02:44.520298 18702 heartbeater.cc:503] Master 127.2.237.62:43887 was elected leader, sending a full tablet report...
I20260620 23:02:44.552176 18842 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:43815
I20260620 23:02:44.552223 18969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:43815 every 8 connection(s)
I20260620 23:02:44.553570 18842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 23:02:44.558130 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 18842
I20260620 23:02:44.558329 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance
I20260620 23:02:44.558874 2996 external_mini_cluster.cc:1428] Setting key 535c7f8b313df6f5f7f44d8c67f029cf
Jun 20 23:02:44 dist-test-slave-f63b krb5kdc[18468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996564, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:02:44.575649 18972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:44.556347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:43143 (local address 127.2.237.62:43887)
0620 23:02:44.556862 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:44.556867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:44.558158 (+ 1291us) server_negotiation.cc:407] Connection header received
0620 23:02:44.560457 (+ 2299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:44.560462 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:44.560554 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:44.560692 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:44.562349 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.563233 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.564815 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.565265 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.565414 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:44.569221 (+ 3807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:44.569226 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:44.569229 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:44.569266 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:44.571935 (+ 2669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:44.572843 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:44.572848 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:44.572851 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:44.573035 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:44.573631 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:44.573636 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:44.573639 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:44.574016 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:44.574201 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:44.575176 (+ 975us) server_negotiation.cc:299] Negotiation successful
0620 23:02:44.575362 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":272,"threads_started":1}
I20260620 23:02:44.576700 18973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:44.556845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.3:43143)
0620 23:02:44.557720 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:44.557768 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:44.560124 (+ 2356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:44.561012 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:44.561022 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:44.561568 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:02:44.562172 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:44.562192 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.563388 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.563393 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:44.564665 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:44.564677 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.565534 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.565543 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:44.565812 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:44.566530 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:02:44.566558 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:02:44.568995 (+ 2437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:02:44.572087 (+ 3092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:44.572095 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:44.572111 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:44.572673 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:44.573187 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:02:44.573192 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:02:44.573195 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:02:44.573490 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:02:44.574183 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:02:44.574189 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:02:44.574770 (+ 581us) client_negotiation.cc:769] Sending connection context
0620 23:02:44.575078 (+ 308us) client_negotiation.cc:240] Negotiation successful
0620 23:02:44.575500 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":355,"threads_started":1}
I20260620 23:02:44.578151 18970 heartbeater.cc:347] Connected to a master server at 127.2.237.62:43887
I20260620 23:02:44.578496 18970 heartbeater.cc:464] Registering TS with master...
I20260620 23:02:44.579286 18970 heartbeater.cc:511] Master 127.2.237.62:43887 requested a full tablet report, sending...
I20260620 23:02:44.580858 18513 ts_manager.cc:194] Registered new tserver with Master: a2ed604e389e43338b71e4b41802334a (127.2.237.3:43815)
I20260620 23:02:44.581822 18513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.3@KRBTEST.COM'} at 127.2.237.3:43143
I20260620 23:02:44.587185 2996 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:02:44.604832 18972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:44.592018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55900 (local address 127.2.237.62:43887)
0620 23:02:44.592213 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:44.592220 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:44.592713 (+ 493us) server_negotiation.cc:407] Connection header received
0620 23:02:44.592802 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:44.592806 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:44.592875 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:44.593039 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:44.594458 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.595351 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.596738 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.597131 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.597243 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:44.599798 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:44.599803 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:44.599806 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:44.599840 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:44.602310 (+ 2470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:44.603109 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:44.603113 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:44.603115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:44.603247 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:44.603582 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:44.603585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:44.603587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:44.603927 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:44.604110 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:44.604399 (+ 289us) server_negotiation.cc:299] Negotiation successful
0620 23:02:44.604574 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260620 23:02:44.609506 18513 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:02:44.616631 18513 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:02:44.642107 18983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:44.629247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:41657 (local address 127.0.0.1:36588)
0620 23:02:44.631296 (+ 2049us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:44.631319 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:44.631454 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:44.633849 (+ 2395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:44.633853 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:44.633890 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:02:44.634606 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:44.634620 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.636698 (+ 2078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.636713 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:44.638831 (+ 2118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:44.638844 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.640874 (+ 2030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.640880 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:44.641143 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:44.641301 (+ 158us) client_negotiation.cc:769] Sending connection context
0620 23:02:44.641414 (+ 113us) client_negotiation.cc:240] Negotiation successful
0620 23:02:44.641666 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1913,"mutex_wait_us":151,"thread_start_us":817,"threads_started":1}
I20260620 23:02:44.642123 18985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:44.629374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57448 (local address 127.2.237.3:43815)
0620 23:02:44.633104 (+ 3730us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:44.633115 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:44.633150 (+ 35us) server_negotiation.cc:407] Connection header received
0620 23:02:44.633247 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:44.633254 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:44.633500 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:44.634320 (+ 820us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:02:44.634695 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.635829 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.639248 (+ 3419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.640616 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.640747 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:44.640962 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:44.641626 (+ 664us) server_negotiation.cc:299] Negotiation successful
0620 23:02:44.641819 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3542,"thread_start_us":3397,"threads_started":1}
I20260620 23:02:44.643028 18986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:44.629863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36588 (local address 127.2.237.1:41657)
0620 23:02:44.633229 (+ 3366us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:44.633238 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:44.633266 (+ 28us) server_negotiation.cc:407] Connection header received
0620 23:02:44.633364 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:44.633371 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:44.633622 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:44.634764 (+ 1142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:02:44.635009 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.636526 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.638989 (+ 2463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.640549 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.640707 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:44.642129 (+ 1422us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:44.642461 (+ 332us) server_negotiation.cc:299] Negotiation successful
0620 23:02:44.642689 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2768,"thread_start_us":2624,"threads_started":1}
I20260620 23:02:44.643194 18984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:44.628945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:43815 (local address 127.0.0.1:57448)
0620 23:02:44.630463 (+ 1518us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:44.630500 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:44.630715 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:44.633728 (+ 3013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:44.633732 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:44.633754 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:02:44.634455 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:44.634472 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.635968 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.635976 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:44.639079 (+ 3103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:44.639095 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.640836 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.640845 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:44.641147 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:44.641293 (+ 146us) client_negotiation.cc:769] Sending connection context
0620 23:02:44.641416 (+ 123us) client_negotiation.cc:240] Negotiation successful
0620 23:02:44.642776 (+ 1360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1285,"mutex_wait_us":1598,"thread_start_us":2060,"threads_started":1}
I20260620 23:02:44.643651 18987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:44.630115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:34287 (local address 127.0.0.1:38720)
0620 23:02:44.632312 (+ 2197us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:44.632325 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:44.632479 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:44.633509 (+ 1030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:44.633516 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:44.633551 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:02:44.634126 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:44.634145 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.636193 (+ 2048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.636198 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:44.639079 (+ 2881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:44.639095 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.642799 (+ 3704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.642807 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:44.642999 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:44.643031 (+ 32us) client_negotiation.cc:769] Sending connection context
0620 23:02:44.643139 (+ 108us) client_negotiation.cc:240] Negotiation successful
0620 23:02:44.643298 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2102,"mutex_wait_us":196,"thread_start_us":1972,"threads_started":1}
I20260620 23:02:44.644407 18988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:44.630795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38720 (local address 127.2.237.2:34287)
0620 23:02:44.631955 (+ 1160us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:44.631963 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:44.632442 (+ 479us) server_negotiation.cc:407] Connection header received
0620 23:02:44.632671 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:44.632683 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:44.632968 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:44.633211 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:02:44.634329 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.636045 (+ 1716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.641067 (+ 5022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.642559 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.642722 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:44.642935 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:44.643882 (+ 947us) server_negotiation.cc:299] Negotiation successful
0620 23:02:44.644066 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1010,"thread_start_us":893,"threads_started":1}
I20260620 23:02:44.646754 18905 tablet_service.cc:1467] Processing CreateTablet for tablet 7db7081740f94d738fdfb44653457d51 (DEFAULT_TABLE table=test-table [id=c664ea2311f54b379d22356a2d3f4652]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:02:44.646770 18637 tablet_service.cc:1467] Processing CreateTablet for tablet 7db7081740f94d738fdfb44653457d51 (DEFAULT_TABLE table=test-table [id=c664ea2311f54b379d22356a2d3f4652]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:02:44.648100 18905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7db7081740f94d738fdfb44653457d51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:02:44.648353 18637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7db7081740f94d738fdfb44653457d51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:02:44.649639 18771 tablet_service.cc:1467] Processing CreateTablet for tablet 7db7081740f94d738fdfb44653457d51 (DEFAULT_TABLE table=test-table [id=c664ea2311f54b379d22356a2d3f4652]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:02:44.650967 18771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7db7081740f94d738fdfb44653457d51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:02:44.660012 18989 tablet_bootstrap.cc:492] T 7db7081740f94d738fdfb44653457d51 P a2ed604e389e43338b71e4b41802334a: Bootstrap starting.
I20260620 23:02:44.664377 18989 tablet_bootstrap.cc:654] T 7db7081740f94d738fdfb44653457d51 P a2ed604e389e43338b71e4b41802334a: Neither blocks nor log segments found. Creating new log.
I20260620 23:02:44.665656 18991 tablet_bootstrap.cc:492] T 7db7081740f94d738fdfb44653457d51 P 42394c1c54b646f08ed66bc1329c070e: Bootstrap starting.
I20260620 23:02:44.665791 18989 log.cc:824] T 7db7081740f94d738fdfb44653457d51 P a2ed604e389e43338b71e4b41802334a: Log is configured to *not* fsync() on all Append() calls
I20260620 23:02:44.667104 18990 tablet_bootstrap.cc:492] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5: Bootstrap starting.
I20260620 23:02:44.671056 18990 tablet_bootstrap.cc:654] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5: Neither blocks nor log segments found. Creating new log.
I20260620 23:02:44.671075 18991 tablet_bootstrap.cc:654] T 7db7081740f94d738fdfb44653457d51 P 42394c1c54b646f08ed66bc1329c070e: Neither blocks nor log segments found. Creating new log.
I20260620 23:02:44.672509 18990 log.cc:824] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5: Log is configured to *not* fsync() on all Append() calls
I20260620 23:02:44.675092 18991 log.cc:824] T 7db7081740f94d738fdfb44653457d51 P 42394c1c54b646f08ed66bc1329c070e: Log is configured to *not* fsync() on all Append() calls
I20260620 23:02:44.676688 18990 tablet_bootstrap.cc:492] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5: No bootstrap required, opened a new log
I20260620 23:02:44.677007 18990 ts_tablet_manager.cc:1397] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.000s
I20260620 23:02:44.677380 18989 tablet_bootstrap.cc:492] T 7db7081740f94d738fdfb44653457d51 P a2ed604e389e43338b71e4b41802334a: No bootstrap required, opened a new log
I20260620 23:02:44.677676 18989 ts_tablet_manager.cc:1397] T 7db7081740f94d738fdfb44653457d51 P a2ed604e389e43338b71e4b41802334a: Time spent bootstrapping tablet: real 0.018s user 0.011s sys 0.004s
I20260620 23:02:44.678201 18991 tablet_bootstrap.cc:492] T 7db7081740f94d738fdfb44653457d51 P 42394c1c54b646f08ed66bc1329c070e: No bootstrap required, opened a new log
I20260620 23:02:44.678480 18991 ts_tablet_manager.cc:1397] T 7db7081740f94d738fdfb44653457d51 P 42394c1c54b646f08ed66bc1329c070e: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.000s
I20260620 23:02:44.685279 18990 raft_consensus.cc:348] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ed604e389e43338b71e4b41802334a" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43815 } } peers { permanent_uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41657 } } peers { permanent_uuid: "42394c1c54b646f08ed66bc1329c070e" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 34287 } }
I20260620 23:02:44.685730 18990 raft_consensus.cc:374] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:02:44.685827 18990 raft_consensus.cc:729] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66cfdaa2f4c143feac2c5a2d76bbdac5, State: Initialized, Role: FOLLOWER
I20260620 23:02:44.686522 18990 consensus_queue.cc:260] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ed604e389e43338b71e4b41802334a" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43815 } } peers { permanent_uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41657 } } peers { permanent_uuid: "42394c1c54b646f08ed66bc1329c070e" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 34287 } }
I20260620 23:02:44.688828 18989 raft_consensus.cc:348] T 7db7081740f94d738fdfb44653457d51 P a2ed604e389e43338b71e4b41802334a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ed604e389e43338b71e4b41802334a" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43815 } } peers { permanent_uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41657 } } peers { permanent_uuid: "42394c1c54b646f08ed66bc1329c070e" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 34287 } }
I20260620 23:02:44.689376 18989 raft_consensus.cc:374] T 7db7081740f94d738fdfb44653457d51 P a2ed604e389e43338b71e4b41802334a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:02:44.689497 18989 raft_consensus.cc:729] T 7db7081740f94d738fdfb44653457d51 P a2ed604e389e43338b71e4b41802334a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ed604e389e43338b71e4b41802334a, State: Initialized, Role: FOLLOWER
I20260620 23:02:44.689662 18990 ts_tablet_manager.cc:1428] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260620 23:02:44.690313 18989 consensus_queue.cc:260] T 7db7081740f94d738fdfb44653457d51 P a2ed604e389e43338b71e4b41802334a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ed604e389e43338b71e4b41802334a" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43815 } } peers { permanent_uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41657 } } peers { permanent_uuid: "42394c1c54b646f08ed66bc1329c070e" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 34287 } }
I20260620 23:02:44.691650 18991 raft_consensus.cc:348] T 7db7081740f94d738fdfb44653457d51 P 42394c1c54b646f08ed66bc1329c070e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ed604e389e43338b71e4b41802334a" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43815 } } peers { permanent_uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41657 } } peers { permanent_uuid: "42394c1c54b646f08ed66bc1329c070e" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 34287 } }
I20260620 23:02:44.692211 18991 raft_consensus.cc:374] T 7db7081740f94d738fdfb44653457d51 P 42394c1c54b646f08ed66bc1329c070e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:02:44.692343 18991 raft_consensus.cc:729] T 7db7081740f94d738fdfb44653457d51 P 42394c1c54b646f08ed66bc1329c070e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42394c1c54b646f08ed66bc1329c070e, State: Initialized, Role: FOLLOWER
I20260620 23:02:44.693045 18991 consensus_queue.cc:260] T 7db7081740f94d738fdfb44653457d51 P 42394c1c54b646f08ed66bc1329c070e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ed604e389e43338b71e4b41802334a" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43815 } } peers { permanent_uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41657 } } peers { permanent_uuid: "42394c1c54b646f08ed66bc1329c070e" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 34287 } }
I20260620 23:02:44.694396 18836 heartbeater.cc:503] Master 127.2.237.62:43887 was elected leader, sending a full tablet report...
I20260620 23:02:44.695096 18991 ts_tablet_manager.cc:1428] T 7db7081740f94d738fdfb44653457d51 P 42394c1c54b646f08ed66bc1329c070e: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
I20260620 23:02:44.696066 18970 heartbeater.cc:503] Master 127.2.237.62:43887 was elected leader, sending a full tablet report...
I20260620 23:02:44.696805 18989 ts_tablet_manager.cc:1428] T 7db7081740f94d738fdfb44653457d51 P a2ed604e389e43338b71e4b41802334a: Time spent starting tablet: real 0.019s user 0.016s sys 0.004s
I20260620 23:02:44.730226 18995 raft_consensus.cc:482] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:02:44.730613 18995 raft_consensus.cc:504] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ed604e389e43338b71e4b41802334a" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43815 } } peers { permanent_uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41657 } } peers { permanent_uuid: "42394c1c54b646f08ed66bc1329c070e" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 34287 } }
I20260620 23:02:44.732411 18995 leader_election.cc:288] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2ed604e389e43338b71e4b41802334a (127.2.237.3:43815), 42394c1c54b646f08ed66bc1329c070e (127.2.237.2:34287)
W20260620 23:02:44.736193 18703 tablet.cc:2369] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:02:44.740187 18988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:44.733012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:34789 (local address 127.2.237.2:34287)
0620 23:02:44.733307 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:44.733314 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:44.733862 (+ 548us) server_negotiation.cc:407] Connection header received
0620 23:02:44.734124 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:44.734132 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:44.734215 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:44.734342 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:02:44.735117 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.736278 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.738256 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.739141 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.739309 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:44.739412 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:44.739792 (+ 380us) server_negotiation.cc:299] Negotiation successful
0620 23:02:44.739921 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260620 23:02:44.740332 18998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:44.732673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:34287 (local address 127.2.237.1:34789)
0620 23:02:44.733714 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:44.733738 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:44.733909 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:44.734358 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:44.734363 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:44.734386 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:02:44.734940 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:44.734958 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.736475 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.736481 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:44.738071 (+ 1590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:44.738085 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.739372 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.739379 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:44.739625 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:44.739653 (+ 28us) client_negotiation.cc:769] Sending connection context
0620 23:02:44.739764 (+ 111us) client_negotiation.cc:240] Negotiation successful
0620 23:02:44.739897 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":860,"mutex_wait_us":78,"thread_start_us":765,"threads_started":1}
I20260620 23:02:44.740260 18985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:44.732881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:32991 (local address 127.2.237.3:43815)
0620 23:02:44.733103 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:44.733113 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:44.733971 (+ 858us) server_negotiation.cc:407] Connection header received
0620 23:02:44.734231 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:44.734236 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:44.734310 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:44.734397 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:02:44.735250 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.736347 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.738361 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.739244 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.739408 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:44.739538 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:44.739880 (+ 342us) server_negotiation.cc:299] Negotiation successful
0620 23:02:44.740014 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 23:02:44.740749 18999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:44.732721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:43815 (local address 127.2.237.1:32991)
0620 23:02:44.733858 (+ 1137us) negotiation.cc:107] Waiting for socket to connect
0620 23:02:44.733876 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 23:02:44.734035 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:02:44.734539 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:02:44.734543 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:02:44.734605 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:02:44.735085 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:44.735098 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.736501 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.736508 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:44.738181 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:02:44.738193 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.739475 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.739481 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:02:44.739701 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:44.739725 (+ 24us) client_negotiation.cc:769] Sending connection context
0620 23:02:44.739843 (+ 118us) client_negotiation.cc:240] Negotiation successful
0620 23:02:44.739965 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":996,"mutex_wait_us":56,"thread_start_us":569,"threads_started":1}
I20260620 23:02:44.741338 18791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7db7081740f94d738fdfb44653457d51" candidate_uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42394c1c54b646f08ed66bc1329c070e" is_pre_election: true
I20260620 23:02:44.741852 18791 raft_consensus.cc:2454] T 7db7081740f94d738fdfb44653457d51 P 42394c1c54b646f08ed66bc1329c070e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66cfdaa2f4c143feac2c5a2d76bbdac5 in term 0.
I20260620 23:02:44.741832 18925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7db7081740f94d738fdfb44653457d51" candidate_uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ed604e389e43338b71e4b41802334a" is_pre_election: true
I20260620 23:02:44.742352 18925 raft_consensus.cc:2454] T 7db7081740f94d738fdfb44653457d51 P a2ed604e389e43338b71e4b41802334a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66cfdaa2f4c143feac2c5a2d76bbdac5 in term 0.
I20260620 23:02:44.742579 18590 leader_election.cc:302] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42394c1c54b646f08ed66bc1329c070e, 66cfdaa2f4c143feac2c5a2d76bbdac5; no voters:
I20260620 23:02:44.743032 18995 raft_consensus.cc:2781] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:02:44.743191 18995 raft_consensus.cc:482] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:02:44.743319 18995 raft_consensus.cc:3037] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:02:44.745517 18995 raft_consensus.cc:504] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ed604e389e43338b71e4b41802334a" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43815 } } peers { permanent_uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41657 } } peers { permanent_uuid: "42394c1c54b646f08ed66bc1329c070e" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 34287 } }
I20260620 23:02:44.746313 18995 leader_election.cc:288] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [CANDIDATE]: Term 1 election: Requested vote from peers a2ed604e389e43338b71e4b41802334a (127.2.237.3:43815), 42394c1c54b646f08ed66bc1329c070e (127.2.237.2:34287)
I20260620 23:02:44.746846 18925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7db7081740f94d738fdfb44653457d51" candidate_uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2ed604e389e43338b71e4b41802334a"
I20260620 23:02:44.746837 18791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7db7081740f94d738fdfb44653457d51" candidate_uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42394c1c54b646f08ed66bc1329c070e"
I20260620 23:02:44.747059 18925 raft_consensus.cc:3037] T 7db7081740f94d738fdfb44653457d51 P a2ed604e389e43338b71e4b41802334a [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:02:44.747059 18791 raft_consensus.cc:3037] T 7db7081740f94d738fdfb44653457d51 P 42394c1c54b646f08ed66bc1329c070e [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:02:44.750406 18791 raft_consensus.cc:2454] T 7db7081740f94d738fdfb44653457d51 P 42394c1c54b646f08ed66bc1329c070e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66cfdaa2f4c143feac2c5a2d76bbdac5 in term 1.
I20260620 23:02:44.750442 18925 raft_consensus.cc:2454] T 7db7081740f94d738fdfb44653457d51 P a2ed604e389e43338b71e4b41802334a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66cfdaa2f4c143feac2c5a2d76bbdac5 in term 1.
I20260620 23:02:44.750939 18590 leader_election.cc:302] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42394c1c54b646f08ed66bc1329c070e, 66cfdaa2f4c143feac2c5a2d76bbdac5; no voters:
I20260620 23:02:44.751313 18995 raft_consensus.cc:2781] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:02:44.751734 18995 raft_consensus.cc:686] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [term 1 LEADER]: Becoming Leader. State: Replica: 66cfdaa2f4c143feac2c5a2d76bbdac5, State: Running, Role: LEADER
I20260620 23:02:44.752226 18995 consensus_queue.cc:237] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ed604e389e43338b71e4b41802334a" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43815 } } peers { permanent_uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41657 } } peers { permanent_uuid: "42394c1c54b646f08ed66bc1329c070e" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 34287 } }
I20260620 23:02:44.757293 18513 catalog_manager.cc:5495] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66cfdaa2f4c143feac2c5a2d76bbdac5 (127.2.237.1). New cstate: current_term: 1 leader_uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ed604e389e43338b71e4b41802334a" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42394c1c54b646f08ed66bc1329c070e" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 34287 } health_report { overall_health: UNKNOWN } } }
W20260620 23:02:44.774930 18837 tablet.cc:2369] T 7db7081740f94d738fdfb44653457d51 P 42394c1c54b646f08ed66bc1329c070e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:02:44.802476 18986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:44.797226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36592 (local address 127.2.237.1:41657)
0620 23:02:44.797421 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:44.797425 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:44.797443 (+ 18us) server_negotiation.cc:407] Connection header received
0620 23:02:44.797553 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:44.797556 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:44.797638 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:44.797771 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:02:44.798163 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.799058 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.800448 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.800838 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.800933 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:44.801453 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:02:44.801990 (+ 537us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:02:44.802133 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:44.802182 (+ 49us) server_negotiation.cc:299] Negotiation successful
0620 23:02:44.802256 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260620 23:02:44.805308 18971 tablet.cc:2369] T 7db7081740f94d738fdfb44653457d51 P a2ed604e389e43338b71e4b41802334a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:02:44.811295 18791 raft_consensus.cc:1261] T 7db7081740f94d738fdfb44653457d51 P 42394c1c54b646f08ed66bc1329c070e [term 1 FOLLOWER]: Refusing update from remote peer 66cfdaa2f4c143feac2c5a2d76bbdac5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:02:44.811298 18925 raft_consensus.cc:1261] T 7db7081740f94d738fdfb44653457d51 P a2ed604e389e43338b71e4b41802334a [term 1 FOLLOWER]: Refusing update from remote peer 66cfdaa2f4c143feac2c5a2d76bbdac5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:02:44.812328 19000 consensus_queue.cc:1035] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42394c1c54b646f08ed66bc1329c070e" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 34287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:02:44.812703 18995 consensus_queue.cc:1035] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2ed604e389e43338b71e4b41802334a" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:02:44.823400 19003 mvcc.cc:205] Tried to move back new op lower bound from 7299057929456680960 to 7299057929229254656. Current Snapshot: MvccSnapshot[applied={T|T < 7299057929456680960}]
I20260620 23:02:44.847285 18513 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:02:44.847604 18513 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:02:44.851570 18513 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05: Sending DeleteTablet for 3 replicas of tablet 7db7081740f94d738fdfb44653457d51
I20260620 23:02:44.852835 18905 tablet_service.cc:1514] Processing DeleteTablet for tablet 7db7081740f94d738fdfb44653457d51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:02:44 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:57448
I20260620 23:02:44.852856 18771 tablet_service.cc:1514] Processing DeleteTablet for tablet 7db7081740f94d738fdfb44653457d51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:02:44 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:38720
I20260620 23:02:44.853574 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 18574
I20260620 23:02:44.853613 18637 tablet_service.cc:1514] Processing DeleteTablet for tablet 7db7081740f94d738fdfb44653457d51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:02:44 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:36588
I20260620 23:02:44.854619 19012 tablet_replica.cc:330] T 7db7081740f94d738fdfb44653457d51 P a2ed604e389e43338b71e4b41802334a: stopping tablet replica
I20260620 23:02:44.855141 19012 raft_consensus.cc:2229] T 7db7081740f94d738fdfb44653457d51 P a2ed604e389e43338b71e4b41802334a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:02:44.855760 19012 raft_consensus.cc:2258] T 7db7081740f94d738fdfb44653457d51 P a2ed604e389e43338b71e4b41802334a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:02:44.856668 19013 tablet_replica.cc:330] T 7db7081740f94d738fdfb44653457d51 P 42394c1c54b646f08ed66bc1329c070e: stopping tablet replica
I20260620 23:02:44.857208 19013 raft_consensus.cc:2229] T 7db7081740f94d738fdfb44653457d51 P 42394c1c54b646f08ed66bc1329c070e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:02:44.857834 19013 raft_consensus.cc:2258] T 7db7081740f94d738fdfb44653457d51 P 42394c1c54b646f08ed66bc1329c070e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:02:44.861656 19012 ts_tablet_manager.cc:1905] T 7db7081740f94d738fdfb44653457d51 P a2ed604e389e43338b71e4b41802334a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:02:44.865958 19013 ts_tablet_manager.cc:1905] T 7db7081740f94d738fdfb44653457d51 P 42394c1c54b646f08ed66bc1329c070e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:02:44.869864 19014 tablet_replica.cc:330] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5: stopping tablet replica
I20260620 23:02:44.874193 19014 raft_consensus.cc:2229] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:02:44.874490 19012 ts_tablet_manager.cc:1918] T 7db7081740f94d738fdfb44653457d51 P a2ed604e389e43338b71e4b41802334a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:02:44.874492 19013 ts_tablet_manager.cc:1918] T 7db7081740f94d738fdfb44653457d51 P 42394c1c54b646f08ed66bc1329c070e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:02:44.874708 19013 log.cc:1196] T 7db7081740f94d738fdfb44653457d51 P 42394c1c54b646f08ed66bc1329c070e: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/wal/wals/7db7081740f94d738fdfb44653457d51
I20260620 23:02:44.874708 19012 log.cc:1196] T 7db7081740f94d738fdfb44653457d51 P a2ed604e389e43338b71e4b41802334a: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/wal/wals/7db7081740f94d738fdfb44653457d51
I20260620 23:02:44.875249 19013 ts_tablet_manager.cc:1939] T 7db7081740f94d738fdfb44653457d51 P 42394c1c54b646f08ed66bc1329c070e: Deleting consensus metadata
I20260620 23:02:44.875249 19012 ts_tablet_manager.cc:1939] T 7db7081740f94d738fdfb44653457d51 P a2ed604e389e43338b71e4b41802334a: Deleting consensus metadata
I20260620 23:02:44.875272 19014 raft_consensus.cc:2258] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:02:44.877235 18497 catalog_manager.cc:4841] TS 42394c1c54b646f08ed66bc1329c070e (127.2.237.2:34287): tablet 7db7081740f94d738fdfb44653457d51 (table test-table [id=c664ea2311f54b379d22356a2d3f4652]) successfully deleted
I20260620 23:02:44.877652 18500 catalog_manager.cc:4841] TS a2ed604e389e43338b71e4b41802334a (127.2.237.3:43815): tablet 7db7081740f94d738fdfb44653457d51 (table test-table [id=c664ea2311f54b379d22356a2d3f4652]) successfully deleted
I20260620 23:02:44.879429 19014 ts_tablet_manager.cc:1905] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5: Deleting tablet data with delete state TABLET_DATA_DELETED
Jun 20 23:02:44 dist-test-slave-f63b krb5kdc[18468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.1@KRBTEST.COM
I20260620 23:02:44.887002 19014 ts_tablet_manager.cc:1918] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:02:44.887195 19014 log.cc:1196] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/wal/wals/7db7081740f94d738fdfb44653457d51
I20260620 23:02:44.887751 19014 ts_tablet_manager.cc:1939] T 7db7081740f94d738fdfb44653457d51 P 66cfdaa2f4c143feac2c5a2d76bbdac5: Deleting consensus metadata
I20260620 23:02:44.889828 18498 catalog_manager.cc:4841] TS 66cfdaa2f4c143feac2c5a2d76bbdac5 (127.2.237.1:41657): tablet 7db7081740f94d738fdfb44653457d51 (table test-table [id=c664ea2311f54b379d22356a2d3f4652]) successfully deleted
I20260620 23:02:44.891072 18986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:44.854919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36600 (local address 127.2.237.1:41657)
0620 23:02:44.855101 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:44.855108 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:44.866096 (+ 10988us) server_negotiation.cc:407] Connection header received
0620 23:02:44.866683 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:44.866689 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:44.866795 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:44.866885 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:44.870166 (+ 3281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.871585 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.875846 (+ 4261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:44.876586 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:44.876693 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:44.882974 (+ 6281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:44.882988 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:44.883007 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:44.883062 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:44.887552 (+ 4490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:44.888546 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:44.888555 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:44.888559 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:44.888760 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:44.889258 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:44.889262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:44.889265 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:44.889835 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:44.890232 (+ 397us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:44.890478 (+ 246us) server_negotiation.cc:299] Negotiation successful
0620 23:02:44.890747 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260620 23:02:44.892140 18697 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:02:44.995141 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 18574
I20260620 23:02:45.013873 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 18708
Jun 20 23:02:45 dist-test-slave-f63b krb5kdc[18468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.2@KRBTEST.COM
I20260620 23:02:45.031810 18988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:45.014627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38728 (local address 127.2.237.2:34287)
0620 23:02:45.014837 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:45.014845 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:45.014932 (+ 87us) server_negotiation.cc:407] Connection header received
0620 23:02:45.015130 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:45.015135 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:45.015217 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:45.015372 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:45.016670 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:45.017419 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:45.018993 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:45.019402 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:45.019555 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:45.024754 (+ 5199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:45.024767 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:45.024793 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:45.024840 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:45.028837 (+ 3997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:45.029577 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:45.029587 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:45.029590 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:45.029738 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:45.030108 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:45.030113 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:45.030116 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:45.030613 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:45.030905 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:45.031205 (+ 300us) server_negotiation.cc:299] Negotiation successful
0620 23:02:45.031443 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260620 23:02:45.032531 18831 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:02:45.126623 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 18708
I20260620 23:02:45.144610 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 18842
Jun 20 23:02:45 dist-test-slave-f63b krb5kdc[18468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.3@KRBTEST.COM
I20260620 23:02:45.160718 18985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:02:45.145601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57452 (local address 127.2.237.3:43815)
0620 23:02:45.145794 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0620 23:02:45.145799 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:02:45.145818 (+ 19us) server_negotiation.cc:407] Connection header received
0620 23:02:45.145963 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:02:45.145967 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:02:45.146035 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:02:45.146117 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:02:45.147659 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:45.148649 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:45.149986 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:02:45.150391 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:02:45.150498 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:02:45.155035 (+ 4537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:02:45.155053 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:02:45.155076 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:02:45.155119 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:02:45.158091 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:45.158813 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:45.158821 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:45.158823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:45.158964 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:02:45.159312 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:02:45.159315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:02:45.159317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:02:45.159653 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:02:45.159872 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0620 23:02:45.160141 (+ 269us) server_negotiation.cc:299] Negotiation successful
0620 23:02:45.160338 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260620 23:02:45.161473 18965 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:02:45.252892 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 18842
I20260620 23:02:45.270761 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 18481
I20260620 23:02:45.271991 18543 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:02:45.387241 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 18481
I20260620 23:03:05.409859 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:43887
--webserver_interface=127.2.237.62
--webserver_port=36741
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:43543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:43887
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:05.763065 19020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:05.763474 19020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:05.763680 19020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:05.771610 19020 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260620 23:03:05.771785 19020 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260620 23:03:05.771873 19020 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260620 23:03:05.771957 19020 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:03:05.772058 19020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:05.772171 19020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:05.772275 19020 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:03:05.772368 19020 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:03:05.783131 19020 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:43543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:43887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:43887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=36741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:03:05.784608 19020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:05.786118 19020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:05.811419 19026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:05.811319 19027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:05.811313 19029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:05.812259 19020 server_base.cc:1034] running on GCE node
I20260620 23:03:05.813271 19020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:05.815119 19020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:05.816385 19020 hybrid_clock.cc:648] HybridClock initialized: now 1781996585816313 us; error 50 us; skew 500 ppm
I20260620 23:03:05.816818 19020 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:03:05 dist-test-slave-f63b krb5kdc[18468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996585, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:05.820785 19020 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 23:03:05.822784 19020 webserver.cc:460] Webserver started at http://127.2.237.62:36741/ using document root <none> and password file <none>
I20260620 23:03:05.823668 19020 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:05.823836 19020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:05.830277 19020 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.001s
I20260620 23:03:05.833527 19036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:05.835258 19020 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260620 23:03:05.835484 19020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "97fc1aadd98c4012b792727ecb1eaf05"
format_stamp: "Formatted at 2026-06-20 23:02:42 on dist-test-slave-f63b"
server_key: "30793aea8b7b5f74a90d93a6700aca15"
server_key_iv: "6c53b05c23f3faf75ef8c113ce2fa582"
server_key_version: "encryptionkey@0"
I20260620 23:03:05.836649 19020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:05.862920 19020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:05.867880 19020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:05.868201 19020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:05.888697 19020 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:43887
I20260620 23:03:05.888738 19087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:43887 every 8 connection(s)
I20260620 23:03:05.890679 19020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:03:05.896118 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 19020
I20260620 23:03:05.897115 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:41657
--local_ip_for_outbound_sockets=127.2.237.1
--tserver_master_addrs=127.2.237.62:43887
--webserver_port=34071
--webserver_interface=127.2.237.1
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:43543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:03:05.900060 19088 sys_catalog.cc:263] Verifying existing consensus state
I20260620 23:03:05.905892 19088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05: Bootstrap starting.
I20260620 23:03:05.927229 19088 log.cc:824] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:05.940443 19088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 23:03:05.941120 19088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05: Bootstrap complete.
I20260620 23:03:05.951128 19088 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97fc1aadd98c4012b792727ecb1eaf05" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43887 } }
I20260620 23:03:05.952127 19088 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97fc1aadd98c4012b792727ecb1eaf05, State: Initialized, Role: FOLLOWER
I20260620 23:03:05.952903 19088 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97fc1aadd98c4012b792727ecb1eaf05" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43887 } }
I20260620 23:03:05.953207 19088 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:03:05.953349 19088 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:03:05.953553 19088 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [term 1 FOLLOWER]: Advancing to term 2
I20260620 23:03:05.957912 19088 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97fc1aadd98c4012b792727ecb1eaf05" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43887 } }
I20260620 23:03:05.958506 19088 leader_election.cc:302] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97fc1aadd98c4012b792727ecb1eaf05; no voters:
I20260620 23:03:05.959262 19088 leader_election.cc:288] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260620 23:03:05.959453 19092 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [term 2 FOLLOWER]: Leader election won for term 2
I20260620 23:03:05.960564 19092 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [term 2 LEADER]: Becoming Leader. State: Replica: 97fc1aadd98c4012b792727ecb1eaf05, State: Running, Role: LEADER
I20260620 23:03:05.961211 19092 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97fc1aadd98c4012b792727ecb1eaf05" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43887 } }
I20260620 23:03:05.961774 19088 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:03:05.963874 19094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97fc1aadd98c4012b792727ecb1eaf05. Latest consensus state: current_term: 2 leader_uuid: "97fc1aadd98c4012b792727ecb1eaf05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97fc1aadd98c4012b792727ecb1eaf05" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43887 } } }
I20260620 23:03:05.964389 19094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:05.965021 19093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "97fc1aadd98c4012b792727ecb1eaf05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97fc1aadd98c4012b792727ecb1eaf05" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43887 } } }
I20260620 23:03:05.965397 19093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:05.967752 19098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:03:05.979254 19098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:03:05.981520 19098 catalog_manager.cc:1259] Loaded cluster ID: 5a37db2db7124067851f01dfc1227a10
I20260620 23:03:05.981751 19098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:03:05.988166 19098 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:03:05.990744 19098 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05: Loaded TSK: 0
I20260620 23:03:06.029098 19110 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 23:03:06.039013 19098 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05: Generated new TSK 1
I20260620 23:03:06.039438 19098 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260620 23:03:06.296937 19090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:06.297307 19090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:06.297482 19090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:06.305305 19090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:06.305472 19090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:06.305665 19090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 23:03:06.316231 19090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:43543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:41657
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=34071
--tserver_master_addrs=127.2.237.62:43887
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:03:06.317718 19090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:06.319185 19090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:06.335669 19116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:06.335851 19119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:06.336376 19117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:06.337075 19090 server_base.cc:1034] running on GCE node
I20260620 23:03:06.337800 19090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:06.338922 19090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:06.340148 19090 hybrid_clock.cc:648] HybridClock initialized: now 1781996586340109 us; error 39 us; skew 500 ppm
I20260620 23:03:06.340502 19090 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:03:06 dist-test-slave-f63b krb5kdc[18468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996586, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:06.348376 19090 init.cc:377] Logged in from keytab as kudu/127.2.237.1@KRBTEST.COM (short username kudu)
I20260620 23:03:06.350145 19090 webserver.cc:460] Webserver started at http://127.2.237.1:34071/ using document root <none> and password file <none>
I20260620 23:03:06.350988 19090 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:06.351161 19090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:06.357134 19090 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 23:03:06.360265 19126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:06.361848 19090 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260620 23:03:06.362072 19090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5"
format_stamp: "Formatted at 2026-06-20 23:02:43 on dist-test-slave-f63b"
server_key: "32a3d3fc08a9f357ad3d7a77590da64f"
server_key_iv: "a141a099226d5fa855cd08f6d93122ba"
server_key_version: "encryptionkey@0"
I20260620 23:03:06.363200 19090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:06.390707 19090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:06.395354 19090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:06.395656 19090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:06.396675 19090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:06.397998 19090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:06.398095 19090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:06.398173 19090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:06.398226 19090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:06.432960 19090 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:41657
I20260620 23:03:06.433068 19238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:41657 every 8 connection(s)
I20260620 23:03:06.434492 19090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 23:03:06.436780 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 19090
I20260620 23:03:06.437628 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:34287
--local_ip_for_outbound_sockets=127.2.237.2
--tserver_master_addrs=127.2.237.62:43887
--webserver_port=46673
--webserver_interface=127.2.237.2
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:43543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 23:03:06 dist-test-slave-f63b krb5kdc[18468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996586, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:03:06.462117 19242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:06.438300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.1:41327)
0620 23:03:06.439356 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:06.439399 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:06.441474 (+ 2075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:06.443217 (+ 1743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:06.443230 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:06.443957 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:06.444549 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:06.444569 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:06.446468 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:06.446480 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:06.447934 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:06.447948 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:06.449179 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:06.449187 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:06.449526 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:06.450357 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:06.450404 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:06.453371 (+ 2967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:06.457331 (+ 3960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:06.457342 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:06.457361 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:06.457851 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:06.458353 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:06.458359 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:06.458363 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:06.458615 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:06.459449 (+ 834us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:06.459458 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:06.459822 (+ 364us) client_negotiation.cc:769] Sending connection context
0620 23:03:06.460095 (+ 273us) client_negotiation.cc:240] Negotiation successful
0620 23:03:06.460714 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":769,"thread_start_us":543,"threads_started":1}
I20260620 23:03:06.462118 19241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:06.438057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:41327 (local address 127.2.237.62:43887)
0620 23:03:06.438788 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:06.438801 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:06.439556 (+ 755us) server_negotiation.cc:407] Connection header received
0620 23:03:06.442137 (+ 2581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:06.442153 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:06.442551 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:06.443052 (+ 501us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:06.444838 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:06.446266 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:06.448112 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:06.448881 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:06.449128 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:06.453699 (+ 4571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:06.453708 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:06.453725 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:06.453780 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:06.457117 (+ 3337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:06.458030 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:06.458039 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:06.458044 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:06.458216 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:06.458763 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:06.458772 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:06.458774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:06.459290 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:06.459516 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:06.460340 (+ 824us) server_negotiation.cc:299] Negotiation successful
0620 23:03:06.460741 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":314,"threads_started":1}
I20260620 23:03:06.465229 19239 heartbeater.cc:347] Connected to a master server at 127.2.237.62:43887
I20260620 23:03:06.465571 19239 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:06.466365 19239 heartbeater.cc:511] Master 127.2.237.62:43887 requested a full tablet report, sending...
I20260620 23:03:06.468766 19053 ts_manager.cc:194] Registered new tserver with Master: 66cfdaa2f4c143feac2c5a2d76bbdac5 (127.2.237.1:41657)
I20260620 23:03:06.470204 19053 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.1@KRBTEST.COM'} at 127.2.237.1:41327
W20260620 23:03:06.783344 19243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:06.783717 19243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:06.783897 19243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:06.791702 19243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:06.791873 19243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:06.792059 19243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 23:03:06.802067 19243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:43543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:34287
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=46673
--tserver_master_addrs=127.2.237.62:43887
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:03:06.803486 19243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:06.804863 19243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:06.818941 19248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:06.819015 19249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:06.819037 19251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:06.820446 19243 server_base.cc:1034] running on GCE node
I20260620 23:03:06.821218 19243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:06.822405 19243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:06.823726 19243 hybrid_clock.cc:648] HybridClock initialized: now 1781996586823655 us; error 78 us; skew 500 ppm
I20260620 23:03:06.824119 19243 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:03:06 dist-test-slave-f63b krb5kdc[18468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996586, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:06.827584 19243 init.cc:377] Logged in from keytab as kudu/127.2.237.2@KRBTEST.COM (short username kudu)
I20260620 23:03:06.829159 19243 webserver.cc:460] Webserver started at http://127.2.237.2:46673/ using document root <none> and password file <none>
I20260620 23:03:06.829879 19243 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:06.830000 19243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:06.835997 19243 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260620 23:03:06.839051 19258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:06.840595 19243 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260620 23:03:06.840797 19243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "42394c1c54b646f08ed66bc1329c070e"
format_stamp: "Formatted at 2026-06-20 23:02:43 on dist-test-slave-f63b"
server_key: "e8b86a2cce2ea4b5a7a698f6cc4b7445"
server_key_iv: "f83acd279c5d076543306ce5be9db94e"
server_key_version: "encryptionkey@0"
I20260620 23:03:06.841917 19243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:06.869779 19243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:06.874410 19243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:06.874735 19243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:06.875840 19243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:06.877244 19243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:06.877344 19243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:06.877494 19243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:06.877576 19243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:06.915127 19243 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:34287
I20260620 23:03:06.915234 19370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:34287 every 8 connection(s)
I20260620 23:03:06.916735 19243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 23:03:06.919440 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 19243
I20260620 23:03:06.920127 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:43815
--local_ip_for_outbound_sockets=127.2.237.3
--tserver_master_addrs=127.2.237.62:43887
--webserver_port=34131
--webserver_interface=127.2.237.3
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:43543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 20 23:03:06 dist-test-slave-f63b krb5kdc[18468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996586, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:03:06.943634 19241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:06.919534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:59875 (local address 127.2.237.62:43887)
0620 23:03:06.919750 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:06.919756 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:06.921110 (+ 1354us) server_negotiation.cc:407] Connection header received
0620 23:03:06.924463 (+ 3353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:06.924468 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:06.924560 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:06.924654 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:06.927244 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:06.927968 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:06.930274 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:06.931093 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:06.931248 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:06.935316 (+ 4068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:06.935326 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:06.935334 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:06.935384 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:06.939301 (+ 3917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:06.940357 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:06.940367 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:06.940373 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:06.940556 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:06.941179 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:06.941188 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:06.941194 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:06.941669 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:06.941870 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:06.942820 (+ 950us) server_negotiation.cc:299] Negotiation successful
0620 23:03:06.943127 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260620 23:03:06.944331 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:06.919970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.2:59875)
0620 23:03:06.920891 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:06.920959 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:06.924055 (+ 3096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:06.925039 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:06.925055 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:06.925994 (+ 939us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:06.927019 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:06.927053 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:06.928215 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:06.928224 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:06.930090 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:06.930106 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:06.931354 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:06.931360 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:06.931594 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:06.932406 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:06.932433 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:06.935028 (+ 2595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:06.939542 (+ 4514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:06.939551 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:06.939568 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:06.940151 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:06.940698 (+ 547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:06.940701 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:06.940704 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:06.941026 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:06.941920 (+ 894us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:06.941928 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:06.942314 (+ 386us) client_negotiation.cc:769] Sending connection context
0620 23:03:06.942627 (+ 313us) client_negotiation.cc:240] Negotiation successful
0620 23:03:06.943124 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":374,"threads_started":1}
I20260620 23:03:06.946240 19371 heartbeater.cc:347] Connected to a master server at 127.2.237.62:43887
I20260620 23:03:06.946682 19371 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:06.947764 19371 heartbeater.cc:511] Master 127.2.237.62:43887 requested a full tablet report, sending...
I20260620 23:03:06.949734 19053 ts_manager.cc:194] Registered new tserver with Master: 42394c1c54b646f08ed66bc1329c070e (127.2.237.2:34287)
I20260620 23:03:06.950768 19053 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.2@KRBTEST.COM'} at 127.2.237.2:59875
W20260620 23:03:07.270097 19374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:07.270397 19374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:07.270592 19374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:07.278079 19374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:07.278199 19374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:07.278388 19374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 23:03:07.288573 19374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:43543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:43815
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=34131
--tserver_master_addrs=127.2.237.62:43887
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:03:07.290195 19374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:07.291622 19374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:07.305406 19382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:07.305434 19379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:07.305397 19380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:07.306136 19374 server_base.cc:1034] running on GCE node
I20260620 23:03:07.306855 19374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:07.307821 19374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:07.309095 19374 hybrid_clock.cc:648] HybridClock initialized: now 1781996587309041 us; error 53 us; skew 500 ppm
I20260620 23:03:07.309445 19374 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:03:07 dist-test-slave-f63b krb5kdc[18468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996587, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:07.312714 19374 init.cc:377] Logged in from keytab as kudu/127.2.237.3@KRBTEST.COM (short username kudu)
I20260620 23:03:07.314306 19374 webserver.cc:460] Webserver started at http://127.2.237.3:34131/ using document root <none> and password file <none>
I20260620 23:03:07.315119 19374 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:07.315269 19374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:07.321126 19374 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260620 23:03:07.324070 19389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:07.325495 19374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260620 23:03:07.325685 19374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "a2ed604e389e43338b71e4b41802334a"
format_stamp: "Formatted at 2026-06-20 23:02:44 on dist-test-slave-f63b"
server_key: "797655a11b17dcdfddde67a64dda03e5"
server_key_iv: "f11e95e0a79d403f4c8e322762e9eea2"
server_key_version: "encryptionkey@0"
I20260620 23:03:07.326792 19374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:07.353040 19374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:07.357312 19374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:07.357628 19374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:07.358829 19374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:07.360314 19374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:07.360447 19374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:07.360574 19374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:07.360666 19374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:07.395077 19374 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:43815
I20260620 23:03:07.395171 19501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:43815 every 8 connection(s)
I20260620 23:03:07.396606 19374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 23:03:07.399852 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 19374
Jun 20 23:03:07 dist-test-slave-f63b krb5kdc[18468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996587, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:03:07.419889 19506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:07.400766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53630 (local address 127.2.237.62:43887)
0620 23:03:07.403413 (+ 2647us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:07.403418 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:07.403971 (+ 553us) server_negotiation.cc:407] Connection header received
0620 23:03:07.404137 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:07.404141 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:07.404196 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:07.404286 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:07.406209 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.407163 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.409031 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.409702 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.409817 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:07.413362 (+ 3545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:07.413368 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:07.413373 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:07.413412 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:07.416673 (+ 3261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:07.417553 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:07.417557 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:07.417559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:07.417692 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:07.418108 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:07.418113 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:07.418116 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:07.418510 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:07.418782 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:07.419178 (+ 396us) server_negotiation.cc:299] Negotiation successful
0620 23:03:07.419414 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2525,"thread_start_us":2407,"threads_started":1}
I20260620 23:03:07.421355 19241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:07.399397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:35381 (local address 127.2.237.62:43887)
0620 23:03:07.399632 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:07.399638 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:07.400965 (+ 1327us) server_negotiation.cc:407] Connection header received
0620 23:03:07.402992 (+ 2027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:07.402998 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:07.403101 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:07.403217 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:07.405649 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.406606 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.408820 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.409438 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.409578 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:07.414246 (+ 4668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:07.414253 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:07.414259 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:07.414335 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:07.417663 (+ 3328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:07.418660 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:07.418665 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:07.418668 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:07.418809 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:07.419358 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:07.419364 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:07.419367 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:07.419703 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:07.419808 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:07.420777 (+ 969us) server_negotiation.cc:299] Negotiation successful
0620 23:03:07.421009 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260620 23:03:07.422541 19504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:07.399946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.3:35381)
0620 23:03:07.400805 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:07.400851 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:07.402694 (+ 1843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:07.403699 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:07.403713 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:07.404539 (+ 826us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:07.405450 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:07.405480 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.406760 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.406763 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:07.408672 (+ 1909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:07.408688 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.409650 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.409655 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:07.409951 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:07.410932 (+ 981us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:07.410969 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:07.413987 (+ 3018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:07.417806 (+ 3819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:07.417819 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:07.417840 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:07.418495 (+ 655us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:07.418939 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:07.418942 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:07.418945 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:07.419212 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:07.419842 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:07.419853 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:07.420303 (+ 450us) client_negotiation.cc:769] Sending connection context
0620 23:03:07.420651 (+ 348us) client_negotiation.cc:240] Negotiation successful
0620 23:03:07.421138 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":371,"threads_started":1}
I20260620 23:03:07.423986 19502 heartbeater.cc:347] Connected to a master server at 127.2.237.62:43887
I20260620 23:03:07.424338 19502 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:07.425161 19502 heartbeater.cc:511] Master 127.2.237.62:43887 requested a full tablet report, sending...
I20260620 23:03:07.426699 19053 ts_manager.cc:194] Registered new tserver with Master: a2ed604e389e43338b71e4b41802334a (127.2.237.3:43815)
I20260620 23:03:07.427551 19053 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.3@KRBTEST.COM'} at 127.2.237.3:35381
I20260620 23:03:07.435485 2996 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:03:07.443177 19241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:07.436494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53642 (local address 127.2.237.62:43887)
0620 23:03:07.436838 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:07.436846 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:07.437207 (+ 361us) server_negotiation.cc:407] Connection header received
0620 23:03:07.437438 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:07.437444 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:07.437610 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:07.437735 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:03:07.438265 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.439133 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.441167 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.441594 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.441735 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:07.442267 (+ 532us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:03:07.442470 (+ 203us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0620 23:03:07.442908 (+ 438us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53642: token expired
Metrics: {"server-negotiator.queue_time_us":180}
W20260620 23:03:07.443332 19241 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53642: token expired
W20260620 23:03:07.443905 18974 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.2.237.62:43887)
I20260620 23:03:07.456806 19241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:07.444423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53646 (local address 127.2.237.62:43887)
0620 23:03:07.444569 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:07.444573 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:07.444588 (+ 15us) server_negotiation.cc:407] Connection header received
0620 23:03:07.444704 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:07.444708 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:07.444767 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:07.444871 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:07.446189 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.447040 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.448343 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.448745 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.448836 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:07.451595 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:07.451600 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:07.451603 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:07.451638 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:07.454309 (+ 2671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:07.455102 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:07.455106 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:07.455109 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:07.455232 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:07.455564 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:07.455567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:07.455569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:07.455881 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:07.456072 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:07.456373 (+ 301us) server_negotiation.cc:299] Negotiation successful
0620 23:03:07.456547 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 23:03:07.460563 19053 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53646:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:03:07.468565 19053 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:03:07.473479 19239 heartbeater.cc:503] Master 127.2.237.62:43887 was elected leader, sending a full tablet report...
I20260620 23:03:07.504484 19513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:07.480925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:43815 (local address 127.0.0.1:38170)
0620 23:03:07.493962 (+ 13037us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:07.494019 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:07.494264 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:07.496294 (+ 2030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:07.496309 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:07.496361 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:07.496911 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:07.496943 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.499154 (+ 2211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.499161 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:07.501600 (+ 2439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:07.501620 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.503415 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.503422 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:07.503636 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:07.503695 (+ 59us) client_negotiation.cc:769] Sending connection context
0620 23:03:07.503840 (+ 145us) client_negotiation.cc:240] Negotiation successful
0620 23:03:07.504055 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12755,"mutex_wait_us":1,"thread_start_us":15352,"threads_started":1}
I20260620 23:03:07.504478 19514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:07.481216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38170 (local address 127.2.237.3:43815)
0620 23:03:07.493047 (+ 11831us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:07.493061 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:07.494527 (+ 1466us) server_negotiation.cc:407] Connection header received
0620 23:03:07.494680 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:07.494692 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:07.495037 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:07.495255 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:07.497244 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.498924 (+ 1680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.501809 (+ 2885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.503188 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.503332 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:07.503592 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:07.503913 (+ 321us) server_negotiation.cc:299] Negotiation successful
0620 23:03:07.504113 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11535,"thread_start_us":11859,"threads_started":1}
I20260620 23:03:07.505884 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:07.483941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60986 (local address 127.2.237.2:34287)
0620 23:03:07.496980 (+ 13039us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:07.496989 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:07.497708 (+ 719us) server_negotiation.cc:407] Connection header received
0620 23:03:07.497815 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:07.497823 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:07.498052 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:07.498279 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:07.499060 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.500661 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.503126 (+ 2465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.504621 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.505069 (+ 448us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:07.505266 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:07.505398 (+ 132us) server_negotiation.cc:299] Negotiation successful
0620 23:03:07.505565 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12859,"thread_start_us":12714,"threads_started":1}
I20260620 23:03:07.505892 19516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:07.483240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:34287 (local address 127.0.0.1:60986)
0620 23:03:07.497449 (+ 14209us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:07.497468 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:07.497588 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:07.498295 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:07.498300 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:07.498366 (+ 66us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:07.498843 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:07.498857 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.500805 (+ 1948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.500810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:07.502977 (+ 2167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:07.502987 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.505030 (+ 2043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.505037 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:07.505221 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:07.505256 (+ 35us) client_negotiation.cc:769] Sending connection context
0620 23:03:07.505341 (+ 85us) client_negotiation.cc:240] Negotiation successful
0620 23:03:07.505472 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14068,"mutex_wait_us":3,"spinlock_wait_cycles":45312,"thread_start_us":13912,"threads_started":1}
I20260620 23:03:07.506932 19512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:07.481050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:41657 (local address 127.0.0.1:51040)
0620 23:03:07.496636 (+ 15586us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:07.496662 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:07.496817 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:07.497856 (+ 1039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:07.497872 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:07.497897 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:07.498734 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:07.498747 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.500416 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.500421 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:07.502717 (+ 2296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:07.502733 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.505201 (+ 2468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.505208 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:07.506381 (+ 1173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:07.506416 (+ 35us) client_negotiation.cc:769] Sending connection context
0620 23:03:07.506497 (+ 81us) client_negotiation.cc:240] Negotiation successful
0620 23:03:07.506645 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15399,"mutex_wait_us":983,"spinlock_wait_cycles":3328,"thread_start_us":12459,"threads_started":1}
I20260620 23:03:07.507721 19515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:07.481562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51040 (local address 127.2.237.1:41657)
0620 23:03:07.496140 (+ 14578us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:07.496153 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:07.496980 (+ 827us) server_negotiation.cc:407] Connection header received
0620 23:03:07.497112 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:07.497122 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:07.497401 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:07.497590 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:07.498988 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.500257 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.502888 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.504363 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.504526 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:07.505029 (+ 503us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:07.507203 (+ 2174us) server_negotiation.cc:299] Negotiation successful
0620 23:03:07.507405 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14358,"thread_start_us":14178,"threads_started":1}
I20260620 23:03:07.509052 19437 tablet_service.cc:1467] Processing CreateTablet for tablet 3a9bc4465fa94fdb83a8e49b88036804 (DEFAULT_TABLE table=test-table [id=0d0aa6eadb854099a54b5c7c72126419]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:07.509641 19306 tablet_service.cc:1467] Processing CreateTablet for tablet 3a9bc4465fa94fdb83a8e49b88036804 (DEFAULT_TABLE table=test-table [id=0d0aa6eadb854099a54b5c7c72126419]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:07.510649 19437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a9bc4465fa94fdb83a8e49b88036804. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:07.510946 19306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a9bc4465fa94fdb83a8e49b88036804. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:07.511225 19174 tablet_service.cc:1467] Processing CreateTablet for tablet 3a9bc4465fa94fdb83a8e49b88036804 (DEFAULT_TABLE table=test-table [id=0d0aa6eadb854099a54b5c7c72126419]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:07.512583 19174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a9bc4465fa94fdb83a8e49b88036804. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:07.522915 19518 tablet_bootstrap.cc:492] T 3a9bc4465fa94fdb83a8e49b88036804 P 42394c1c54b646f08ed66bc1329c070e: Bootstrap starting.
I20260620 23:03:07.526957 19518 tablet_bootstrap.cc:654] T 3a9bc4465fa94fdb83a8e49b88036804 P 42394c1c54b646f08ed66bc1329c070e: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:07.527392 19519 tablet_bootstrap.cc:492] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a: Bootstrap starting.
I20260620 23:03:07.528421 19518 log.cc:824] T 3a9bc4465fa94fdb83a8e49b88036804 P 42394c1c54b646f08ed66bc1329c070e: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:07.530398 19520 tablet_bootstrap.cc:492] T 3a9bc4465fa94fdb83a8e49b88036804 P 66cfdaa2f4c143feac2c5a2d76bbdac5: Bootstrap starting.
I20260620 23:03:07.531263 19519 tablet_bootstrap.cc:654] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:07.532272 19518 tablet_bootstrap.cc:492] T 3a9bc4465fa94fdb83a8e49b88036804 P 42394c1c54b646f08ed66bc1329c070e: No bootstrap required, opened a new log
I20260620 23:03:07.532560 19518 ts_tablet_manager.cc:1397] T 3a9bc4465fa94fdb83a8e49b88036804 P 42394c1c54b646f08ed66bc1329c070e: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260620 23:03:07.532622 19519 log.cc:824] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:07.534471 19520 tablet_bootstrap.cc:654] T 3a9bc4465fa94fdb83a8e49b88036804 P 66cfdaa2f4c143feac2c5a2d76bbdac5: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:07.535676 19520 log.cc:824] T 3a9bc4465fa94fdb83a8e49b88036804 P 66cfdaa2f4c143feac2c5a2d76bbdac5: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:07.537177 19519 tablet_bootstrap.cc:492] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a: No bootstrap required, opened a new log
I20260620 23:03:07.537528 19519 ts_tablet_manager.cc:1397] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260620 23:03:07.539645 19520 tablet_bootstrap.cc:492] T 3a9bc4465fa94fdb83a8e49b88036804 P 66cfdaa2f4c143feac2c5a2d76bbdac5: No bootstrap required, opened a new log
I20260620 23:03:07.539989 19520 ts_tablet_manager.cc:1397] T 3a9bc4465fa94fdb83a8e49b88036804 P 66cfdaa2f4c143feac2c5a2d76bbdac5: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260620 23:03:07.541949 19518 raft_consensus.cc:348] T 3a9bc4465fa94fdb83a8e49b88036804 P 42394c1c54b646f08ed66bc1329c070e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ed604e389e43338b71e4b41802334a" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43815 } } peers { permanent_uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41657 } } peers { permanent_uuid: "42394c1c54b646f08ed66bc1329c070e" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 34287 } }
I20260620 23:03:07.542361 19518 raft_consensus.cc:374] T 3a9bc4465fa94fdb83a8e49b88036804 P 42394c1c54b646f08ed66bc1329c070e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:07.542457 19518 raft_consensus.cc:729] T 3a9bc4465fa94fdb83a8e49b88036804 P 42394c1c54b646f08ed66bc1329c070e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42394c1c54b646f08ed66bc1329c070e, State: Initialized, Role: FOLLOWER
I20260620 23:03:07.543170 19518 consensus_queue.cc:260] T 3a9bc4465fa94fdb83a8e49b88036804 P 42394c1c54b646f08ed66bc1329c070e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ed604e389e43338b71e4b41802334a" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43815 } } peers { permanent_uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41657 } } peers { permanent_uuid: "42394c1c54b646f08ed66bc1329c070e" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 34287 } }
I20260620 23:03:07.547540 19371 heartbeater.cc:503] Master 127.2.237.62:43887 was elected leader, sending a full tablet report...
I20260620 23:03:07.548141 19518 ts_tablet_manager.cc:1428] T 3a9bc4465fa94fdb83a8e49b88036804 P 42394c1c54b646f08ed66bc1329c070e: Time spent starting tablet: real 0.015s user 0.010s sys 0.006s
I20260620 23:03:07.550163 19519 raft_consensus.cc:348] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ed604e389e43338b71e4b41802334a" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43815 } } peers { permanent_uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41657 } } peers { permanent_uuid: "42394c1c54b646f08ed66bc1329c070e" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 34287 } }
I20260620 23:03:07.550674 19519 raft_consensus.cc:374] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:07.550786 19519 raft_consensus.cc:729] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2ed604e389e43338b71e4b41802334a, State: Initialized, Role: FOLLOWER
I20260620 23:03:07.550791 19520 raft_consensus.cc:348] T 3a9bc4465fa94fdb83a8e49b88036804 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ed604e389e43338b71e4b41802334a" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43815 } } peers { permanent_uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41657 } } peers { permanent_uuid: "42394c1c54b646f08ed66bc1329c070e" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 34287 } }
I20260620 23:03:07.551273 19520 raft_consensus.cc:374] T 3a9bc4465fa94fdb83a8e49b88036804 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:07.551426 19520 raft_consensus.cc:729] T 3a9bc4465fa94fdb83a8e49b88036804 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66cfdaa2f4c143feac2c5a2d76bbdac5, State: Initialized, Role: FOLLOWER
I20260620 23:03:07.552047 19520 consensus_queue.cc:260] T 3a9bc4465fa94fdb83a8e49b88036804 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ed604e389e43338b71e4b41802334a" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43815 } } peers { permanent_uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41657 } } peers { permanent_uuid: "42394c1c54b646f08ed66bc1329c070e" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 34287 } }
I20260620 23:03:07.551538 19519 consensus_queue.cc:260] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ed604e389e43338b71e4b41802334a" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43815 } } peers { permanent_uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41657 } } peers { permanent_uuid: "42394c1c54b646f08ed66bc1329c070e" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 34287 } }
I20260620 23:03:07.554193 19520 ts_tablet_manager.cc:1428] T 3a9bc4465fa94fdb83a8e49b88036804 P 66cfdaa2f4c143feac2c5a2d76bbdac5: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260620 23:03:07.555332 19502 heartbeater.cc:503] Master 127.2.237.62:43887 was elected leader, sending a full tablet report...
I20260620 23:03:07.555927 19519 ts_tablet_manager.cc:1428] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a: Time spent starting tablet: real 0.018s user 0.013s sys 0.005s
W20260620 23:03:07.648332 19503 tablet.cc:2369] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:03:07.668771 19372 tablet.cc:2369] T 3a9bc4465fa94fdb83a8e49b88036804 P 42394c1c54b646f08ed66bc1329c070e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:03:07.684207 19526 raft_consensus.cc:482] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:03:07.684572 19526 raft_consensus.cc:504] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ed604e389e43338b71e4b41802334a" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43815 } } peers { permanent_uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41657 } } peers { permanent_uuid: "42394c1c54b646f08ed66bc1329c070e" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 34287 } }
I20260620 23:03:07.686447 19526 leader_election.cc:288] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66cfdaa2f4c143feac2c5a2d76bbdac5 (127.2.237.1:41657), 42394c1c54b646f08ed66bc1329c070e (127.2.237.2:34287)
W20260620 23:03:07.687022 19240 tablet.cc:2369] T 3a9bc4465fa94fdb83a8e49b88036804 P 66cfdaa2f4c143feac2c5a2d76bbdac5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:03:07.693285 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:07.687246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:36925 (local address 127.2.237.2:34287)
0620 23:03:07.687748 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:07.687754 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:07.687773 (+ 19us) server_negotiation.cc:407] Connection header received
0620 23:03:07.687852 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:07.687857 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:07.687952 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:07.688077 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:07.688512 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.689442 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.691460 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.692339 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.692490 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:07.692588 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:07.692896 (+ 308us) server_negotiation.cc:299] Negotiation successful
0620 23:03:07.693029 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351}
I20260620 23:03:07.693326 19504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:07.686737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:34287 (local address 127.2.237.3:36925)
0620 23:03:07.686967 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:07.686990 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:07.687128 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:07.688115 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:07.688120 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:07.688148 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:07.688346 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:07.688354 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.689575 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.689582 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:07.691264 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:07.691277 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.692520 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.692526 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:07.692723 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:07.692751 (+ 28us) client_negotiation.cc:769] Sending connection context
0620 23:03:07.692860 (+ 109us) client_negotiation.cc:240] Negotiation successful
0620 23:03:07.693004 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":1}
I20260620 23:03:07.693388 19515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:07.687305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:47731 (local address 127.2.237.1:41657)
0620 23:03:07.687516 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:07.687522 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:07.687806 (+ 284us) server_negotiation.cc:407] Connection header received
0620 23:03:07.688008 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:07.688014 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:07.688088 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:07.688208 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:07.688813 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.689530 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.691583 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.692435 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.692592 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:07.692699 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:07.693006 (+ 307us) server_negotiation.cc:299] Negotiation successful
0620 23:03:07.693136 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260620 23:03:07.693540 19527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:07.686826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:41657 (local address 127.2.237.3:47731)
0620 23:03:07.687693 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:07.687708 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:07.687855 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:07.688225 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:07.688229 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:07.688256 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:07.688640 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:07.688648 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.689679 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.689685 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:07.691377 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:07.691411 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.692659 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.692664 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:07.692839 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:07.692861 (+ 22us) client_negotiation.cc:769] Sending connection context
0620 23:03:07.692977 (+ 116us) client_negotiation.cc:240] Negotiation successful
0620 23:03:07.693110 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"mutex_wait_us":43,"thread_start_us":541,"threads_started":1}
I20260620 23:03:07.694810 19326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a9bc4465fa94fdb83a8e49b88036804" candidate_uuid: "a2ed604e389e43338b71e4b41802334a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42394c1c54b646f08ed66bc1329c070e" is_pre_election: true
I20260620 23:03:07.694818 19194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a9bc4465fa94fdb83a8e49b88036804" candidate_uuid: "a2ed604e389e43338b71e4b41802334a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5" is_pre_election: true
I20260620 23:03:07.695453 19326 raft_consensus.cc:2454] T 3a9bc4465fa94fdb83a8e49b88036804 P 42394c1c54b646f08ed66bc1329c070e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ed604e389e43338b71e4b41802334a in term 0.
I20260620 23:03:07.695441 19194 raft_consensus.cc:2454] T 3a9bc4465fa94fdb83a8e49b88036804 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2ed604e389e43338b71e4b41802334a in term 0.
I20260620 23:03:07.696214 19390 leader_election.cc:302] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 42394c1c54b646f08ed66bc1329c070e, 66cfdaa2f4c143feac2c5a2d76bbdac5, a2ed604e389e43338b71e4b41802334a; no voters:
I20260620 23:03:07.696671 19526 raft_consensus.cc:2781] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:03:07.696825 19526 raft_consensus.cc:482] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:03:07.696934 19526 raft_consensus.cc:3037] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:07.699252 19526 raft_consensus.cc:504] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ed604e389e43338b71e4b41802334a" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43815 } } peers { permanent_uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41657 } } peers { permanent_uuid: "42394c1c54b646f08ed66bc1329c070e" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 34287 } }
I20260620 23:03:07.700003 19526 leader_election.cc:288] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a [CANDIDATE]: Term 1 election: Requested vote from peers 66cfdaa2f4c143feac2c5a2d76bbdac5 (127.2.237.1:41657), 42394c1c54b646f08ed66bc1329c070e (127.2.237.2:34287)
I20260620 23:03:07.700459 19194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a9bc4465fa94fdb83a8e49b88036804" candidate_uuid: "a2ed604e389e43338b71e4b41802334a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5"
I20260620 23:03:07.700611 19326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a9bc4465fa94fdb83a8e49b88036804" candidate_uuid: "a2ed604e389e43338b71e4b41802334a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42394c1c54b646f08ed66bc1329c070e"
I20260620 23:03:07.700696 19194 raft_consensus.cc:3037] T 3a9bc4465fa94fdb83a8e49b88036804 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:07.700861 19326 raft_consensus.cc:3037] T 3a9bc4465fa94fdb83a8e49b88036804 P 42394c1c54b646f08ed66bc1329c070e [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:07.703043 19194 raft_consensus.cc:2454] T 3a9bc4465fa94fdb83a8e49b88036804 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ed604e389e43338b71e4b41802334a in term 1.
I20260620 23:03:07.703168 19326 raft_consensus.cc:2454] T 3a9bc4465fa94fdb83a8e49b88036804 P 42394c1c54b646f08ed66bc1329c070e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2ed604e389e43338b71e4b41802334a in term 1.
I20260620 23:03:07.703508 19391 leader_election.cc:302] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66cfdaa2f4c143feac2c5a2d76bbdac5, a2ed604e389e43338b71e4b41802334a; no voters:
I20260620 23:03:07.703850 19526 raft_consensus.cc:2781] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:07.704309 19526 raft_consensus.cc:686] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a [term 1 LEADER]: Becoming Leader. State: Replica: a2ed604e389e43338b71e4b41802334a, State: Running, Role: LEADER
I20260620 23:03:07.704851 19526 consensus_queue.cc:237] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ed604e389e43338b71e4b41802334a" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43815 } } peers { permanent_uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41657 } } peers { permanent_uuid: "42394c1c54b646f08ed66bc1329c070e" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 34287 } }
I20260620 23:03:07.709699 19052 catalog_manager.cc:5495] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a reported cstate change: term changed from 0 to 1, leader changed from <none> to a2ed604e389e43338b71e4b41802334a (127.2.237.3). New cstate: current_term: 1 leader_uuid: "a2ed604e389e43338b71e4b41802334a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2ed604e389e43338b71e4b41802334a" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 43815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42394c1c54b646f08ed66bc1329c070e" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 34287 } health_report { overall_health: UNKNOWN } } }
I20260620 23:03:07.744616 19514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:07.739248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38174 (local address 127.2.237.3:43815)
0620 23:03:07.739427 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:07.739432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:07.739531 (+ 99us) server_negotiation.cc:407] Connection header received
0620 23:03:07.739735 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:07.739739 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:07.739817 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:07.740015 (+ 198us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:03:07.740261 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.741111 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.742483 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.742980 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.743083 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:07.743551 (+ 468us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:03:07.744121 (+ 570us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:03:07.744242 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:07.744328 (+ 86us) server_negotiation.cc:299] Negotiation successful
0620 23:03:07.744411 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 23:03:07.753471 19326 raft_consensus.cc:1261] T 3a9bc4465fa94fdb83a8e49b88036804 P 42394c1c54b646f08ed66bc1329c070e [term 1 FOLLOWER]: Refusing update from remote peer a2ed604e389e43338b71e4b41802334a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:07.753471 19194 raft_consensus.cc:1261] T 3a9bc4465fa94fdb83a8e49b88036804 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [term 1 FOLLOWER]: Refusing update from remote peer a2ed604e389e43338b71e4b41802334a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:07.754637 19528 consensus_queue.cc:1035] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a [LEADER]: Connected to new peer: Peer: permanent_uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 41657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:03:07.755028 19526 consensus_queue.cc:1035] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a [LEADER]: Connected to new peer: Peer: permanent_uuid: "42394c1c54b646f08ed66bc1329c070e" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 34287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:03:07.772998 19532 mvcc.cc:205] Tried to move back new op lower bound from 7299058023427366912 to 7299058023242731520. Current Snapshot: MvccSnapshot[applied={T|T < 7299058023427366912}]
I20260620 23:03:07.782868 19053 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:03:07.783291 19053 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:03:07.787443 19053 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05: Sending DeleteTablet for 3 replicas of tablet 3a9bc4465fa94fdb83a8e49b88036804
I20260620 23:03:07.788744 19174 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a9bc4465fa94fdb83a8e49b88036804 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:07 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:51040
I20260620 23:03:07.788771 19437 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a9bc4465fa94fdb83a8e49b88036804 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:07 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:38170
I20260620 23:03:07.789569 19306 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a9bc4465fa94fdb83a8e49b88036804 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:07 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:60986
I20260620 23:03:07.789755 19541 tablet_replica.cc:330] T 3a9bc4465fa94fdb83a8e49b88036804 P 66cfdaa2f4c143feac2c5a2d76bbdac5: stopping tablet replica
I20260620 23:03:07.790232 19541 raft_consensus.cc:2229] T 3a9bc4465fa94fdb83a8e49b88036804 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:03:07.790983 19541 raft_consensus.cc:2258] T 3a9bc4465fa94fdb83a8e49b88036804 P 66cfdaa2f4c143feac2c5a2d76bbdac5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:07.791998 19542 tablet_replica.cc:330] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a: stopping tablet replica
I20260620 23:03:07.792085 19543 tablet_replica.cc:330] T 3a9bc4465fa94fdb83a8e49b88036804 P 42394c1c54b646f08ed66bc1329c070e: stopping tablet replica
I20260620 23:03:07.792466 19542 raft_consensus.cc:2229] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:03:07.792555 19543 raft_consensus.cc:2229] T 3a9bc4465fa94fdb83a8e49b88036804 P 42394c1c54b646f08ed66bc1329c070e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:03:07.793113 19543 raft_consensus.cc:2258] T 3a9bc4465fa94fdb83a8e49b88036804 P 42394c1c54b646f08ed66bc1329c070e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:07.793221 19542 raft_consensus.cc:2258] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:07.794008 19541 ts_tablet_manager.cc:1905] T 3a9bc4465fa94fdb83a8e49b88036804 P 66cfdaa2f4c143feac2c5a2d76bbdac5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:07.795236 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 19090
I20260620 23:03:07.797140 19543 ts_tablet_manager.cc:1905] T 3a9bc4465fa94fdb83a8e49b88036804 P 42394c1c54b646f08ed66bc1329c070e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:07.800971 19541 ts_tablet_manager.cc:1918] T 3a9bc4465fa94fdb83a8e49b88036804 P 66cfdaa2f4c143feac2c5a2d76bbdac5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:03:07.801226 19541 log.cc:1196] T 3a9bc4465fa94fdb83a8e49b88036804 P 66cfdaa2f4c143feac2c5a2d76bbdac5: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/wal/wals/3a9bc4465fa94fdb83a8e49b88036804
I20260620 23:03:07.801838 19541 ts_tablet_manager.cc:1939] T 3a9bc4465fa94fdb83a8e49b88036804 P 66cfdaa2f4c143feac2c5a2d76bbdac5: Deleting consensus metadata
I20260620 23:03:07.803946 19038 catalog_manager.cc:4841] TS 66cfdaa2f4c143feac2c5a2d76bbdac5 (127.2.237.1:41657): tablet 3a9bc4465fa94fdb83a8e49b88036804 (table test-table [id=0d0aa6eadb854099a54b5c7c72126419]) successfully deleted
I20260620 23:03:07.804317 19543 ts_tablet_manager.cc:1918] T 3a9bc4465fa94fdb83a8e49b88036804 P 42394c1c54b646f08ed66bc1329c070e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:03:07.804587 19543 log.cc:1196] T 3a9bc4465fa94fdb83a8e49b88036804 P 42394c1c54b646f08ed66bc1329c070e: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/wal/wals/3a9bc4465fa94fdb83a8e49b88036804
I20260620 23:03:07.805238 19543 ts_tablet_manager.cc:1939] T 3a9bc4465fa94fdb83a8e49b88036804 P 42394c1c54b646f08ed66bc1329c070e: Deleting consensus metadata
I20260620 23:03:07.805603 19542 ts_tablet_manager.cc:1905] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:07.807334 19037 catalog_manager.cc:4841] TS 42394c1c54b646f08ed66bc1329c070e (127.2.237.2:34287): tablet 3a9bc4465fa94fdb83a8e49b88036804 (table test-table [id=0d0aa6eadb854099a54b5c7c72126419]) successfully deleted
I20260620 23:03:07.813484 19542 ts_tablet_manager.cc:1918] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:03:07.813773 19542 log.cc:1196] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/wal/wals/3a9bc4465fa94fdb83a8e49b88036804
I20260620 23:03:07.814268 19542 ts_tablet_manager.cc:1939] T 3a9bc4465fa94fdb83a8e49b88036804 P a2ed604e389e43338b71e4b41802334a: Deleting consensus metadata
I20260620 23:03:07.815564 19515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:07.796411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51042 (local address 127.2.237.1:41657)
0620 23:03:07.796567 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:07.796573 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:07.796592 (+ 19us) server_negotiation.cc:407] Connection header received
0620 23:03:07.796649 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:07.796655 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:07.796725 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:07.796818 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:07.799022 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.800232 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.802392 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.803058 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.803182 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:07.807058 (+ 3876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:07.807074 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:07.807094 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:07.807144 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:07.811202 (+ 4058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:07.812238 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:07.812250 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:07.812254 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:07.812429 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:07.812878 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:07.812883 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:07.812886 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:07.813366 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:07.813691 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:07.814905 (+ 1214us) server_negotiation.cc:299] Negotiation successful
0620 23:03:07.815165 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 23:03:07.815948 19040 catalog_manager.cc:4841] TS a2ed604e389e43338b71e4b41802334a (127.2.237.3:43815): tablet 3a9bc4465fa94fdb83a8e49b88036804 (table test-table [id=0d0aa6eadb854099a54b5c7c72126419]) successfully deleted
I20260620 23:03:07.816826 19234 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:03:07.909950 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 19090
I20260620 23:03:07.933359 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 19243
I20260620 23:03:07.955704 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:07.934214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60996 (local address 127.2.237.2:34287)
0620 23:03:07.934418 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:07.934424 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:07.934446 (+ 22us) server_negotiation.cc:407] Connection header received
0620 23:03:07.934659 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:07.934665 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:07.934773 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:07.934885 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:07.936701 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.937900 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.940061 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:07.940774 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:07.940906 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:07.949228 (+ 8322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:07.949245 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:07.949265 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:07.949308 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:07.952569 (+ 3261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:07.953544 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:07.953551 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:07.953554 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:07.953693 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:07.954121 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:07.954124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:07.954126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:07.954538 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:07.954849 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:07.955209 (+ 360us) server_negotiation.cc:299] Negotiation successful
0620 23:03:07.955403 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260620 23:03:07.956453 19366 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:03:08.047262 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 19243
I20260620 23:03:08.065030 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 19374
I20260620 23:03:08.082151 19514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:08.065856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38188 (local address 127.2.237.3:43815)
0620 23:03:08.066099 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:08.066106 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:08.066131 (+ 25us) server_negotiation.cc:407] Connection header received
0620 23:03:08.066230 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:08.066235 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:08.066336 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:08.066463 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:08.067979 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:08.069223 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:08.070641 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:08.071328 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:08.071500 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:08.074131 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:08.074149 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:08.074170 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:08.074222 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:08.078624 (+ 4402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:08.079605 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:08.079616 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:08.079620 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:08.079838 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:08.080291 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:08.080298 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:08.080301 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:08.080781 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:08.081175 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:08.081466 (+ 291us) server_negotiation.cc:299] Negotiation successful
0620 23:03:08.081709 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260620 23:03:08.083153 19497 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:03:08.209858 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 19374
I20260620 23:03:08.231827 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 19020
I20260620 23:03:08.233018 19083 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:03:08.351043 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 19020
I20260620 23:03:28.373682 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:43887
--webserver_interface=127.2.237.62
--webserver_port=36741
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:43543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:43887
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:28.703779 19548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:28.704129 19548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:28.704308 19548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:28.713724 19548 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260620 23:03:28.713869 19548 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260620 23:03:28.713945 19548 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260620 23:03:28.714035 19548 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:03:28.714111 19548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:28.714208 19548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:28.714318 19548 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:03:28.714399 19548 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:03:28.724664 19548 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:43543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:43887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:43887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=36741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:03:28.726068 19548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:28.727699 19548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:28.740522 19553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:28.740589 19556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:28.740548 19554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:28.741031 19548 server_base.cc:1034] running on GCE node
I20260620 23:03:28.741978 19548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:28.743474 19548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:28.744722 19548 hybrid_clock.cc:648] HybridClock initialized: now 1781996608744663 us; error 39 us; skew 500 ppm
I20260620 23:03:28.745069 19548 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:03:28 dist-test-slave-f63b krb5kdc[18468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996608, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:28.751727 19548 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 23:03:28.753320 19548 webserver.cc:460] Webserver started at http://127.2.237.62:36741/ using document root <none> and password file <none>
I20260620 23:03:28.754045 19548 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:28.754137 19548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:28.759958 19548 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260620 23:03:28.762916 19563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:28.764422 19548 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260620 23:03:28.764572 19548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "97fc1aadd98c4012b792727ecb1eaf05"
format_stamp: "Formatted at 2026-06-20 23:02:42 on dist-test-slave-f63b"
server_key: "30793aea8b7b5f74a90d93a6700aca15"
server_key_iv: "6c53b05c23f3faf75ef8c113ce2fa582"
server_key_version: "encryptionkey@0"
I20260620 23:03:28.765543 19548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:28.785594 19548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:28.790970 19548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:28.791263 19548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:28.811821 19548 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:43887
I20260620 23:03:28.811856 19614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:43887 every 8 connection(s)
I20260620 23:03:28.813556 19548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:03:28.818503 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 19548
I20260620 23:03:28.819540 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:41657
--local_ip_for_outbound_sockets=127.2.237.1
--tserver_master_addrs=127.2.237.62:43887
--webserver_port=34071
--webserver_interface=127.2.237.1
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:43543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:03:28.821872 19615 sys_catalog.cc:263] Verifying existing consensus state
I20260620 23:03:28.827348 19615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05: Bootstrap starting.
I20260620 23:03:28.849025 19615 log.cc:824] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:28.865192 19615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260620 23:03:28.865777 19615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05: Bootstrap complete.
I20260620 23:03:28.875468 19615 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97fc1aadd98c4012b792727ecb1eaf05" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43887 } }
I20260620 23:03:28.876406 19615 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97fc1aadd98c4012b792727ecb1eaf05, State: Initialized, Role: FOLLOWER
I20260620 23:03:28.877031 19615 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97fc1aadd98c4012b792727ecb1eaf05" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43887 } }
I20260620 23:03:28.877272 19615 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:03:28.877398 19615 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:03:28.877602 19615 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [term 2 FOLLOWER]: Advancing to term 3
I20260620 23:03:28.881707 19615 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97fc1aadd98c4012b792727ecb1eaf05" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43887 } }
I20260620 23:03:28.882205 19615 leader_election.cc:302] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97fc1aadd98c4012b792727ecb1eaf05; no voters:
I20260620 23:03:28.882966 19615 leader_election.cc:288] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260620 23:03:28.883116 19619 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [term 3 FOLLOWER]: Leader election won for term 3
I20260620 23:03:28.884146 19619 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [term 3 LEADER]: Becoming Leader. State: Replica: 97fc1aadd98c4012b792727ecb1eaf05, State: Running, Role: LEADER
I20260620 23:03:28.884785 19619 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97fc1aadd98c4012b792727ecb1eaf05" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43887 } }
I20260620 23:03:28.885186 19615 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:03:28.888412 19621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97fc1aadd98c4012b792727ecb1eaf05. Latest consensus state: current_term: 3 leader_uuid: "97fc1aadd98c4012b792727ecb1eaf05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97fc1aadd98c4012b792727ecb1eaf05" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43887 } } }
I20260620 23:03:28.888793 19621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:28.889243 19620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "97fc1aadd98c4012b792727ecb1eaf05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97fc1aadd98c4012b792727ecb1eaf05" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43887 } } }
I20260620 23:03:28.889539 19620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:28.903540 19626 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:03:28.910800 19626 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:03:28.911917 19626 catalog_manager.cc:1259] Loaded cluster ID: 5a37db2db7124067851f01dfc1227a10
I20260620 23:03:28.912037 19626 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:03:28.914880 19626 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:03:28.916059 19626 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05: Loaded TSK: 0
I20260620 23:03:28.916208 19626 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 97fc1aadd98c4012b792727ecb1eaf05: Loaded TSK: 1
F20260620 23:03:28.917069 19626 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe6b0f649ad google::LogMessage::Fail() at ??:0
@ 0x7fe6b0f6788a google::LogMessage::SendToLog() at ??:0
@ 0x7fe6b0f644de google::LogMessage::Flush() at ??:0
@ 0x7fe6b0f65a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe6b4cc87d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fe6b4cb773e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fe6c69931c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fe6c6992783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fe6c6a1e2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fe6c6a1e0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fe6c6a685e9 std::function<>::operator()() at ??:0
@ 0x7fe6c6996cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fe6c6994c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe6c6a1c610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fe6c6a1c439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fe6b3bb0345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe6b3bb391c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fe6b3bb3799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fe6b3b8eeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe6b6e686db start_thread at ??:0
@ 0x7fe6afdfe61f clone at ??:0
W20260620 23:03:29.232692 19617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:29.232985 19617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:29.233080 19617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:29.240610 19617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:29.240736 19617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:29.240859 19617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 23:03:29.250941 19617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:43543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:41657
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=34071
--tserver_master_addrs=127.2.237.62:43887
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:03:29.252286 19617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:29.253674 19617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:29.268172 19662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:29.268229 19665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:29.268432 19663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:29.269902 19617 server_base.cc:1034] running on GCE node
I20260620 23:03:29.270547 19617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:29.271617 19617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:29.272890 19617 hybrid_clock.cc:648] HybridClock initialized: now 1781996609272815 us; error 40 us; skew 500 ppm
I20260620 23:03:29.273422 19617 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:03:29 dist-test-slave-f63b krb5kdc[18468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996609, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:29.277446 19617 init.cc:377] Logged in from keytab as kudu/127.2.237.1@KRBTEST.COM (short username kudu)
I20260620 23:03:29.279290 19617 webserver.cc:460] Webserver started at http://127.2.237.1:34071/ using document root <none> and password file <none>
I20260620 23:03:29.280166 19617 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:29.280285 19617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:29.286798 19617 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260620 23:03:29.290091 19672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:29.291855 19617 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260620 23:03:29.292038 19617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "66cfdaa2f4c143feac2c5a2d76bbdac5"
format_stamp: "Formatted at 2026-06-20 23:02:43 on dist-test-slave-f63b"
server_key: "32a3d3fc08a9f357ad3d7a77590da64f"
server_key_iv: "a141a099226d5fa855cd08f6d93122ba"
server_key_version: "encryptionkey@0"
I20260620 23:03:29.293089 19617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:29.311538 19617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:29.316345 19617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:29.316604 19617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:29.317641 19617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:29.319248 19617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:29.319350 19617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:29.319437 19617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:29.319494 19617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:29.355551 19617 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:41657
I20260620 23:03:29.355654 19784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:41657 every 8 connection(s)
I20260620 23:03:29.357172 19617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 23:03:29.358565 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 19617
I20260620 23:03:29.359356 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:34287
--local_ip_for_outbound_sockets=127.2.237.2
--tserver_master_addrs=127.2.237.62:43887
--webserver_port=46673
--webserver_interface=127.2.237.2
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:43543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:03:29.363210 19787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:29.360862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.1:42867)
0620 23:03:29.361803 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:29.362089 (+ 286us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":441,"threads_started":1}
W20260620 23:03:29.364055 19785 heartbeater.cc:644] Failed to heartbeat to 127.2.237.62:43887 (0 consecutive failures): Network error: Failed to ping master at 127.2.237.62:43887: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
I20260620 23:03:29.365108 19787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:29.364687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.1:55135)
0620 23:03:29.364876 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:29.364981 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260620 23:03:29.366397 19787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:29.365834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.1:50403)
0620 23:03:29.366046 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:29.366164 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260620 23:03:29.366709 19785 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260620 23:03:29.705971 19788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:29.706314 19788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:29.706501 19788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:29.714028 19788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:29.714201 19788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:29.714391 19788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 23:03:29.724737 19788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:43543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:34287
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=46673
--tserver_master_addrs=127.2.237.62:43887
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:03:29.726140 19788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:29.727598 19788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:29.741132 19793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:29.741173 19794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:29.741218 19796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:29.742702 19788 server_base.cc:1034] running on GCE node
I20260620 23:03:29.743360 19788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:29.744421 19788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:29.745666 19788 hybrid_clock.cc:648] HybridClock initialized: now 1781996609745582 us; error 54 us; skew 500 ppm
I20260620 23:03:29.746100 19788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:03:29 dist-test-slave-f63b krb5kdc[18468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996609, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:29.751178 19788 init.cc:377] Logged in from keytab as kudu/127.2.237.2@KRBTEST.COM (short username kudu)
I20260620 23:03:29.753187 19788 webserver.cc:460] Webserver started at http://127.2.237.2:46673/ using document root <none> and password file <none>
I20260620 23:03:29.754101 19788 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:29.754272 19788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:29.760064 19788 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260620 23:03:29.762835 19803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:29.764243 19788 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260620 23:03:29.764389 19788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "42394c1c54b646f08ed66bc1329c070e"
format_stamp: "Formatted at 2026-06-20 23:02:43 on dist-test-slave-f63b"
server_key: "e8b86a2cce2ea4b5a7a698f6cc4b7445"
server_key_iv: "f83acd279c5d076543306ce5be9db94e"
server_key_version: "encryptionkey@0"
I20260620 23:03:29.765359 19788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:29.793004 19788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:29.797618 19788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:29.797858 19788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:29.799075 19788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:29.800460 19788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:29.800549 19788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:29.800624 19788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:29.800678 19788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:29.834367 19788 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:34287
I20260620 23:03:29.834471 19915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:34287 every 8 connection(s)
I20260620 23:03:29.835886 19788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 23:03:29.840911 19918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:29.839008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.2:49247)
0620 23:03:29.839745 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:29.839997 (+ 252us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":279,"threads_started":1}
W20260620 23:03:29.841593 19916 heartbeater.cc:644] Failed to heartbeat to 127.2.237.62:43887 (0 consecutive failures): Network error: Failed to ping master at 127.2.237.62:43887: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
I20260620 23:03:29.842424 19918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:29.842108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.2:47847)
0620 23:03:29.842247 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:29.842313 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260620 23:03:29.843216 19918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:29.842918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.2:54187)
0620 23:03:29.843035 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:29.843103 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260620 23:03:29.843466 19916 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 23:03:29.845759 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 19788
I20260620 23:03:29.846387 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:43815
--local_ip_for_outbound_sockets=127.2.237.3
--tserver_master_addrs=127.2.237.62:43887
--webserver_port=34131
--webserver_interface=127.2.237.3
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:43543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:30.174928 19919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:30.175238 19919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:30.175482 19919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:30.183341 19919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:30.183578 19919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:30.183756 19919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 23:03:30.194065 19919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:43543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:43815
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=34131
--tserver_master_addrs=127.2.237.62:43887
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:03:30.195493 19919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:30.196864 19919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:30.211015 19925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:30.211040 19924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:30.211086 19927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:30.212205 19919 server_base.cc:1034] running on GCE node
I20260620 23:03:30.212877 19919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:30.213881 19919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:30.215106 19919 hybrid_clock.cc:648] HybridClock initialized: now 1781996610215069 us; error 32 us; skew 500 ppm
I20260620 23:03:30.215427 19919 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:03:30 dist-test-slave-f63b krb5kdc[18468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996610, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:30.218549 19919 init.cc:377] Logged in from keytab as kudu/127.2.237.3@KRBTEST.COM (short username kudu)
I20260620 23:03:30.220045 19919 webserver.cc:460] Webserver started at http://127.2.237.3:34131/ using document root <none> and password file <none>
I20260620 23:03:30.220791 19919 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:30.220949 19919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:30.226348 19919 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260620 23:03:30.229267 19934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:30.230809 19919 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 23:03:30.230993 19919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "a2ed604e389e43338b71e4b41802334a"
format_stamp: "Formatted at 2026-06-20 23:02:44 on dist-test-slave-f63b"
server_key: "797655a11b17dcdfddde67a64dda03e5"
server_key_iv: "f11e95e0a79d403f4c8e322762e9eea2"
server_key_version: "encryptionkey@0"
I20260620 23:03:30.232004 19919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:30.255620 19919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:30.260206 19919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:30.260515 19919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:30.261906 19919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:30.263890 19919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:30.264019 19919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:30.264137 19919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:30.264212 19919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:30.297540 19919 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:43815
I20260620 23:03:30.297633 20046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:43815 every 8 connection(s)
I20260620 23:03:30.299053 19919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 23:03:30.300576 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 19919
I20260620 23:03:30.304039 20050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:30.302117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.3:33105)
0620 23:03:30.302909 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:30.303167 (+ 258us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":328,"threads_started":1}
W20260620 23:03:30.304693 20047 heartbeater.cc:644] Failed to heartbeat to 127.2.237.62:43887 (0 consecutive failures): Network error: Failed to ping master at 127.2.237.62:43887: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
I20260620 23:03:30.305434 20050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:30.305130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.3:40341)
0620 23:03:30.305264 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:30.305327 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260620 23:03:30.306226 20050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:30.305912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.3:39547)
0620 23:03:30.306047 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:30.306126 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260620 23:03:30.306439 20047 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260620 23:03:30.368198 20051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:30.367314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.1:44679)
0620 23:03:30.367834 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:30.367947 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":259,"threads_started":1}
I20260620 23:03:30.845121 20052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:30.844071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.2:33653)
0620 23:03:30.844774 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:30.844864 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":283,"threads_started":1}
I20260620 23:03:31.307885 20053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:31.307062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.3:35289)
0620 23:03:31.307565 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:31.307657 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":261,"threads_started":1}
I20260620 23:03:31.369997 20054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:31.369268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.1:58679)
0620 23:03:31.369723 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:31.369813 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":223,"threads_started":1}
I20260620 23:03:31.846974 20055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:31.846149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.2:39423)
0620 23:03:31.846705 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:31.846794 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":244,"threads_started":1}
I20260620 23:03:32.309578 20056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:32.308848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.3:51813)
0620 23:03:32.309323 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:32.309411 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":239,"threads_started":1}
I20260620 23:03:32.371743 20057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:32.371026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.1:36735)
0620 23:03:32.371486 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:32.371568 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":232,"threads_started":1}
I20260620 23:03:32.848783 20058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:32.847933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.2:49533)
0620 23:03:32.848479 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:32.848572 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":294,"threads_started":1}
I20260620 23:03:33.311381 20059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:33.310673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.3:56261)
0620 23:03:33.311112 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:33.311194 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":240,"threads_started":1}
I20260620 23:03:33.373374 20060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:33.372636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.1:50761)
0620 23:03:33.373114 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:33.373201 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":252,"threads_started":1}
I20260620 23:03:33.850418 20061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:33.849706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.2:45907)
0620 23:03:33.850142 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:33.850235 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":217,"threads_started":1}
I20260620 23:03:34.313040 20062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:34.312319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.3:44687)
0620 23:03:34.312766 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:34.312851 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":227,"threads_started":1}
I20260620 23:03:34.375207 20063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:34.374442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.1:42431)
0620 23:03:34.374945 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:34.375035 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":279,"threads_started":1}
I20260620 23:03:34.852264 20064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:34.851497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.2:60439)
0620 23:03:34.852000 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:34.852086 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":260,"threads_started":1}
I20260620 23:03:35.314863 20065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:35.314060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.3:45735)
0620 23:03:35.314538 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:35.314685 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":237,"threads_started":1}
I20260620 23:03:35.376904 20066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:35.376207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.1:59479)
0620 23:03:35.376637 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:35.376739 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":227,"threads_started":1}
I20260620 23:03:35.854167 20067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:35.853368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.2:42057)
0620 23:03:35.853886 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:35.854000 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":268,"threads_started":1}
I20260620 23:03:36.316794 20068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:36.315829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.3:37477)
0620 23:03:36.316433 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:36.316562 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":285,"threads_started":1}
I20260620 23:03:36.378721 20069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:36.377870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.1:44447)
0620 23:03:36.378409 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:36.378491 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":243,"threads_started":1}
I20260620 23:03:36.856146 20070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:36.855228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.2:57129)
0620 23:03:36.855813 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:36.855896 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":244,"threads_started":1}
I20260620 23:03:37.318681 20071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:37.317776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.3:55183)
0620 23:03:37.318295 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:37.318390 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":218,"threads_started":1}
I20260620 23:03:37.380613 20072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:37.379718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.1:37339)
0620 23:03:37.380275 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:37.380386 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":236,"threads_started":1}
I20260620 23:03:37.858008 20073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:37.857244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.2:37289)
0620 23:03:37.857753 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:37.857837 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":253,"threads_started":1}
I20260620 23:03:38.320425 20074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:38.319688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.3:34883)
0620 23:03:38.320168 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:38.320256 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":228,"threads_started":1}
I20260620 23:03:38.382313 20075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:38.381573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.1:46757)
0620 23:03:38.382048 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:38.382130 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":230,"threads_started":1}
I20260620 23:03:38.859944 20076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:38.859157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.2:48259)
0620 23:03:38.859671 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:38.859763 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":243,"threads_started":1}
I20260620 23:03:39.322247 20077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:39.321445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.3:59371)
0620 23:03:39.321974 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:39.322061 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":286,"threads_started":1}
I20260620 23:03:39.384069 20078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:39.383333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.1:39539)
0620 23:03:39.383806 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:39.383888 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":270,"threads_started":1}
I20260620 23:03:39.861882 20079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:39.861100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.2:52517)
0620 23:03:39.861628 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:39.861717 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":304,"threads_started":1}
I20260620 23:03:40.323874 20080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:40.323161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.3:53023)
0620 23:03:40.323643 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:40.323718 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":246,"threads_started":1}
I20260620 23:03:40.385958 20081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:40.385202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.1:34445)
0620 23:03:40.385677 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:40.385777 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":246,"threads_started":1}
I20260620 23:03:40.863781 20082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:40.862989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.2:50115)
0620 23:03:40.863489 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:40.863584 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":279,"threads_started":1}
I20260620 23:03:41.325707 20083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:41.324920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.3:48079)
0620 23:03:41.325382 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:41.325494 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":228,"threads_started":1}
I20260620 23:03:41.387709 20084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:41.387035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.1:56395)
0620 23:03:41.387474 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:41.387553 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":230,"threads_started":1}
I20260620 23:03:41.865432 20085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:41.864737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.2:43005)
0620 23:03:41.865188 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:41.865272 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":212,"threads_started":1}
I20260620 23:03:42.327420 20086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:42.326728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.3:37737)
0620 23:03:42.327176 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:42.327265 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":235,"threads_started":1}
I20260620 23:03:42.389725 20087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:42.388746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.1:52011)
0620 23:03:42.389374 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:42.389476 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":277,"threads_started":1}
I20260620 23:03:42.867270 20088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:42.866430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.2:53135)
0620 23:03:42.866975 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:42.867070 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":300,"threads_started":1}
I20260620 23:03:43.329250 20089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:43.328442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.3:38083)
0620 23:03:43.328935 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:43.329051 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":268,"threads_started":1}
I20260620 23:03:43.391606 20090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:43.390834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.1:33365)
0620 23:03:43.391294 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:43.391383 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":219,"threads_started":1}
I20260620 23:03:43.869143 20091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:43.868365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.2:36501)
0620 23:03:43.868872 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:43.868960 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":248,"threads_started":1}
I20260620 23:03:44.331108 20092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:44.330321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.3:49107)
0620 23:03:44.330828 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:44.330914 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":304,"threads_started":1}
I20260620 23:03:44.393494 20093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:44.392752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.1:42791)
0620 23:03:44.393204 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:44.393300 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":233,"threads_started":1}
I20260620 23:03:44.871035 20094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:44.870218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.2:36811)
0620 23:03:44.870726 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:44.870853 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":239,"threads_started":1}
I20260620 23:03:45.306496 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 19617
I20260620 23:03:45.325565 20095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:45.307910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43064 (local address 127.2.237.1:41657)
0620 23:03:45.308597 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:45.308619 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:45.308667 (+ 48us) server_negotiation.cc:407] Connection header received
0620 23:03:45.310424 (+ 1757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:45.310434 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:45.310964 (+ 530us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:45.311331 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:45.312916 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:45.314142 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:45.315504 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:45.316169 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:45.316366 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:45.319074 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:45.319081 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:45.319096 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:45.319135 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:45.321972 (+ 2837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:45.322714 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:45.322723 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:45.322739 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:45.322956 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:45.323423 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:45.323429 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:45.323435 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:45.324097 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:45.324410 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:45.324850 (+ 440us) server_negotiation.cc:299] Negotiation successful
0620 23:03:45.325290 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":303,"threads_started":1}
I20260620 23:03:45.327092 19780 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:03:45.332978 20097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:45.332164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.3:51659)
0620 23:03:45.332679 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:45.332797 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":254,"threads_started":1}
I20260620 23:03:45.411252 20098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:45.410090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43887 (local address 127.2.237.1:43163)
0620 23:03:45.410713 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:45.410914 (+ 201us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.62:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":300,"threads_started":1}
I20260620 23:03:45.411582 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 19617
I20260620 23:03:45.429557 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 19788
I20260620 23:03:45.447556 20099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:45.430526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50544 (local address 127.2.237.2:34287)
0620 23:03:45.431186 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:45.431201 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:45.431238 (+ 37us) server_negotiation.cc:407] Connection header received
0620 23:03:45.432791 (+ 1553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:45.432801 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:45.433240 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:45.433598 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:45.435196 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:45.436474 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:45.437765 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:45.438427 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:45.438663 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:45.441213 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:45.441220 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:45.441235 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:45.441292 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:45.444186 (+ 2894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:45.444865 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:45.444872 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:45.444878 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:45.445147 (+ 269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:45.445522 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:45.445527 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:45.445530 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:45.446055 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:45.446415 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:45.446861 (+ 446us) server_negotiation.cc:299] Negotiation successful
0620 23:03:45.447275 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":288,"threads_started":1}
I20260620 23:03:45.448974 19911 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:03:45.531776 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 19788
I20260620 23:03:45.549223 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 19919
I20260620 23:03:45.567762 20101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:45.550386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40804 (local address 127.2.237.3:43815)
0620 23:03:45.551043 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:45.551058 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:45.551102 (+ 44us) server_negotiation.cc:407] Connection header received
0620 23:03:45.552713 (+ 1611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:45.552723 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:45.553147 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:45.553450 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:45.554890 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:45.556213 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:45.557596 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:45.558268 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:45.558404 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:45.561305 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:45.561313 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:45.561328 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:45.561370 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:45.564519 (+ 3149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:45.565181 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:45.565188 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:45.565195 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:45.565368 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:45.565724 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:45.565727 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:45.565729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:45.566247 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:45.566468 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:45.567075 (+ 607us) server_negotiation.cc:299] Negotiation successful
0620 23:03:45.567419 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":336,"threads_started":1}
I20260620 23:03:45.569178 20042 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:03:45.652895 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 19919
2026-06-20T23:03:45Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64658 ms)
[----------] 30 tests from SecurityITest (306808 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:03:45 dist-test-slave-f63b krb5kdc[20105](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:03:45 dist-test-slave-f63b krb5kdc[20105](info): set up 2 sockets
Jun 20 23:03:45 dist-test-slave-f63b krb5kdc[20105](info): commencing operation
krb5kdc: starting...
W20260620 23:03:46.762651 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.001s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 23:03:46 dist-test-slave-f63b krb5kdc[20105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:03:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:03:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
I20260620 23:03:47.016253 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:38561
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:40939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:38561
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:47.349042 20118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:47.349332 20118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:47.349529 20118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:47.356956 20118 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:03:47.357064 20118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:47.357146 20118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:47.357251 20118 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:03:47.357329 20118 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:03:47.367313 20118 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:40939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:38561
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:38561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:03:47.368758 20118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:47.370121 20118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:47.382597 20126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:47.382711 20124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:47.382839 20123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:47.383571 20118 server_base.cc:1034] running on GCE node
I20260620 23:03:47.384513 20118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:47.386029 20118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:47.387296 20118 hybrid_clock.cc:648] HybridClock initialized: now 1781996627387254 us; error 52 us; skew 500 ppm
I20260620 23:03:47.387689 20118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:03:47 dist-test-slave-f63b krb5kdc[20105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996627, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:47.395658 20118 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 23:03:47.397161 20118 webserver.cc:460] Webserver started at http://127.2.237.62:43177/ using document root <none> and password file <none>
I20260620 23:03:47.397835 20118 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:47.397936 20118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:47.398231 20118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:47.401379 20118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "759fd70d56f74ccba53bdd0ff8964755"
format_stamp: "Formatted at 2026-06-20 23:03:47 on dist-test-slave-f63b"
server_key: "659022c5ac4c70786e07e556ccea17e4"
server_key_iv: "d6c9d11db4fcc065769bf13b1e622523"
server_key_version: "encryptionkey@0"
I20260620 23:03:47.402100 20118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "759fd70d56f74ccba53bdd0ff8964755"
format_stamp: "Formatted at 2026-06-20 23:03:47 on dist-test-slave-f63b"
server_key: "659022c5ac4c70786e07e556ccea17e4"
server_key_iv: "d6c9d11db4fcc065769bf13b1e622523"
server_key_version: "encryptionkey@0"
I20260620 23:03:47.407420 20118 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260620 23:03:47.410930 20133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:47.412293 20118 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260620 23:03:47.412437 20118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "759fd70d56f74ccba53bdd0ff8964755"
format_stamp: "Formatted at 2026-06-20 23:03:47 on dist-test-slave-f63b"
server_key: "659022c5ac4c70786e07e556ccea17e4"
server_key_iv: "d6c9d11db4fcc065769bf13b1e622523"
server_key_version: "encryptionkey@0"
I20260620 23:03:47.412606 20118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:47.426657 20118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:47.431051 20118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:47.431303 20118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:47.449838 20118 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:38561
I20260620 23:03:47.449858 20184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:38561 every 8 connection(s)
I20260620 23:03:47.451381 20118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:03:47.455698 20185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:47.460507 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 20118
I20260620 23:03:47.460752 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:03:47.461387 2996 external_mini_cluster.cc:1428] Setting key 4fba08ef86665a52442dcf7ce6c03dce
I20260620 23:03:47.466429 20185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 759fd70d56f74ccba53bdd0ff8964755: Bootstrap starting.
I20260620 23:03:47.470204 20185 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 759fd70d56f74ccba53bdd0ff8964755: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:47.471452 20185 log.cc:824] T 00000000000000000000000000000000 P 759fd70d56f74ccba53bdd0ff8964755: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:47.474910 20185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 759fd70d56f74ccba53bdd0ff8964755: No bootstrap required, opened a new log
Jun 20 23:03:47 dist-test-slave-f63b krb5kdc[20105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:03:47.485613 20185 raft_consensus.cc:348] T 00000000000000000000000000000000 P 759fd70d56f74ccba53bdd0ff8964755 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "759fd70d56f74ccba53bdd0ff8964755" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 38561 } }
I20260620 23:03:47.486114 20185 raft_consensus.cc:374] T 00000000000000000000000000000000 P 759fd70d56f74ccba53bdd0ff8964755 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:47.486224 20185 raft_consensus.cc:729] T 00000000000000000000000000000000 P 759fd70d56f74ccba53bdd0ff8964755 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 759fd70d56f74ccba53bdd0ff8964755, State: Initialized, Role: FOLLOWER
I20260620 23:03:47.486769 20188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:47.463399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49422 (local address 127.2.237.62:38561)
0620 23:03:47.464220 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:47.464234 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:47.464282 (+ 48us) server_negotiation.cc:407] Connection header received
0620 23:03:47.465992 (+ 1710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:47.466017 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:47.466399 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:47.466816 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:47.468619 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:47.469882 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:47.471841 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:47.472961 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:47.473095 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:47.477774 (+ 4679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:47.477784 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:47.477802 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:47.477849 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:47.481968 (+ 4119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:47.482849 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:47.482858 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:47.482869 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:47.483092 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:47.483475 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:47.483480 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:47.483483 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:47.484249 (+ 766us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:47.484579 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:47.484863 (+ 284us) server_negotiation.cc:299] Negotiation successful
0620 23:03:47.485319 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":389,"threads_started":1}
I20260620 23:03:47.486949 20185 consensus_queue.cc:260] T 00000000000000000000000000000000 P 759fd70d56f74ccba53bdd0ff8964755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "759fd70d56f74ccba53bdd0ff8964755" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 38561 } }
I20260620 23:03:47.487244 20185 raft_consensus.cc:388] T 00000000000000000000000000000000 P 759fd70d56f74ccba53bdd0ff8964755 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:03:47.487417 20185 raft_consensus.cc:482] T 00000000000000000000000000000000 P 759fd70d56f74ccba53bdd0ff8964755 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:03:47.487628 20185 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 759fd70d56f74ccba53bdd0ff8964755 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:47.489979 20185 raft_consensus.cc:504] T 00000000000000000000000000000000 P 759fd70d56f74ccba53bdd0ff8964755 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "759fd70d56f74ccba53bdd0ff8964755" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 38561 } }
I20260620 23:03:47.490468 20185 leader_election.cc:302] T 00000000000000000000000000000000 P 759fd70d56f74ccba53bdd0ff8964755 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 759fd70d56f74ccba53bdd0ff8964755; no voters:
I20260620 23:03:47.491092 20185 leader_election.cc:288] T 00000000000000000000000000000000 P 759fd70d56f74ccba53bdd0ff8964755 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:03:47.491237 20190 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 759fd70d56f74ccba53bdd0ff8964755 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:47.491971 20190 raft_consensus.cc:686] T 00000000000000000000000000000000 P 759fd70d56f74ccba53bdd0ff8964755 [term 1 LEADER]: Becoming Leader. State: Replica: 759fd70d56f74ccba53bdd0ff8964755, State: Running, Role: LEADER
I20260620 23:03:47.492560 20190 consensus_queue.cc:237] T 00000000000000000000000000000000 P 759fd70d56f74ccba53bdd0ff8964755 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "759fd70d56f74ccba53bdd0ff8964755" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 38561 } }
I20260620 23:03:47.493144 20185 sys_catalog.cc:564] T 00000000000000000000000000000000 P 759fd70d56f74ccba53bdd0ff8964755 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:03:47.495548 20192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 759fd70d56f74ccba53bdd0ff8964755 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 759fd70d56f74ccba53bdd0ff8964755. Latest consensus state: current_term: 1 leader_uuid: "759fd70d56f74ccba53bdd0ff8964755" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "759fd70d56f74ccba53bdd0ff8964755" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 38561 } } }
I20260620 23:03:47.495846 20192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 759fd70d56f74ccba53bdd0ff8964755 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:47.496824 20197 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:03:47.496744 20191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 759fd70d56f74ccba53bdd0ff8964755 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "759fd70d56f74ccba53bdd0ff8964755" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "759fd70d56f74ccba53bdd0ff8964755" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 38561 } } }
I20260620 23:03:47.497138 20191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 759fd70d56f74ccba53bdd0ff8964755 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:47.506228 20197 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:03:47.516451 20197 catalog_manager.cc:1347] Generated new cluster ID: acaa46d9351441feb05c9e295d62d205
I20260620 23:03:47.516621 20197 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:03:47.528708 20197 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:03:47.530004 20197 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:03:47.537321 20197 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 759fd70d56f74ccba53bdd0ff8964755: Generated new TSK 0
I20260620 23:03:47.538364 20197 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.237.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
I20260620 23:03:47.610467 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:0
--local_ip_for_outbound_sockets=127.2.237.1
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:38561
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:40939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:47.943818 20211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:47.944144 20211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:47.944327 20211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:47.952080 20211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:47.952234 20211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:47.952417 20211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 23:03:47.962543 20211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:40939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:38561
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:03:47.964011 20211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:47.965427 20211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:47.979449 20216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:47.979393 20217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:47.979365 20219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:47.980275 20211 server_base.cc:1034] running on GCE node
I20260620 23:03:47.980858 20211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:47.981763 20211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:47.982956 20211 hybrid_clock.cc:648] HybridClock initialized: now 1781996627982894 us; error 61 us; skew 500 ppm
I20260620 23:03:47.983317 20211 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:03:47 dist-test-slave-f63b krb5kdc[20105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996627, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:47.986963 20211 init.cc:377] Logged in from keytab as kudu/127.2.237.1@KRBTEST.COM (short username kudu)
I20260620 23:03:47.988461 20211 webserver.cc:460] Webserver started at http://127.2.237.1:43467/ using document root <none> and password file <none>
I20260620 23:03:47.989122 20211 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:47.989218 20211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:47.989470 20211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:47.992384 20211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-0/data/instance:
uuid: "d137499136d9442b96caedae7d6902c8"
format_stamp: "Formatted at 2026-06-20 23:03:47 on dist-test-slave-f63b"
server_key: "8471ab5ede65f39eec793b26227e8deb"
server_key_iv: "ab84346640cd6fb28dcb260d50e5fce2"
server_key_version: "encryptionkey@0"
I20260620 23:03:47.993180 20211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance:
uuid: "d137499136d9442b96caedae7d6902c8"
format_stamp: "Formatted at 2026-06-20 23:03:47 on dist-test-slave-f63b"
server_key: "8471ab5ede65f39eec793b26227e8deb"
server_key_iv: "ab84346640cd6fb28dcb260d50e5fce2"
server_key_version: "encryptionkey@0"
I20260620 23:03:47.998230 20211 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260620 23:03:48.001798 20226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:48.003266 20211 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 23:03:48.003443 20211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "d137499136d9442b96caedae7d6902c8"
format_stamp: "Formatted at 2026-06-20 23:03:47 on dist-test-slave-f63b"
server_key: "8471ab5ede65f39eec793b26227e8deb"
server_key_iv: "ab84346640cd6fb28dcb260d50e5fce2"
server_key_version: "encryptionkey@0"
I20260620 23:03:48.003654 20211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:48.030725 20211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:48.034818 20211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:48.035107 20211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:48.036075 20211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:48.037606 20211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:48.037708 20211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:48.037854 20211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:48.037927 20211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:48.071076 20211 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:34371
I20260620 23:03:48.071158 20338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:34371 every 8 connection(s)
I20260620 23:03:48.072381 20211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 23:03:48.078189 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 20211
I20260620 23:03:48.078418 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance
I20260620 23:03:48.079160 2996 external_mini_cluster.cc:1428] Setting key ae5b8174f44fd9b4c653110c0854a7c1
Jun 20 23:03:48 dist-test-slave-f63b krb5kdc[20105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996627, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:03:48.092211 20341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:48.075028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:51315 (local address 127.2.237.62:38561)
0620 23:03:48.075503 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:48.075508 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:48.076341 (+ 833us) server_negotiation.cc:407] Connection header received
0620 23:03:48.078195 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:48.078200 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:48.078338 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:48.078451 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:48.080023 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:48.080789 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:48.082359 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:48.082854 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:48.082995 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:48.086378 (+ 3383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:48.086391 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:48.086394 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:48.086426 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:48.089111 (+ 2685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:48.089828 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:48.089835 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:48.089839 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:48.089964 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:48.090378 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:48.090385 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:48.090390 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:48.090751 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:48.090917 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:48.091624 (+ 707us) server_negotiation.cc:299] Negotiation successful
0620 23:03:48.091839 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":249,"threads_started":1}
I20260620 23:03:48.093271 20342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:48.075447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:38561 (local address 127.2.237.1:51315)
0620 23:03:48.076170 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:48.076214 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:48.077903 (+ 1689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:48.078722 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:48.078732 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:48.079240 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:48.079843 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:48.079859 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:48.080964 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:48.080968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:48.082214 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:48.082226 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:48.083093 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:48.083104 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:48.083336 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:48.084008 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:48.084032 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:48.086165 (+ 2133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:48.089266 (+ 3101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:48.089275 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:48.089291 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:48.089707 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:48.090080 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:48.090085 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:48.090089 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:48.090268 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:48.090878 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:48.090887 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:48.091256 (+ 369us) client_negotiation.cc:769] Sending connection context
0620 23:03:48.091535 (+ 279us) client_negotiation.cc:240] Negotiation successful
0620 23:03:48.092067 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":284,"threads_started":1}
I20260620 23:03:48.095039 20339 heartbeater.cc:347] Connected to a master server at 127.2.237.62:38561
I20260620 23:03:48.095343 20339 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:48.096042 20339 heartbeater.cc:511] Master 127.2.237.62:38561 requested a full tablet report, sending...
I20260620 23:03:48.098124 20150 ts_manager.cc:194] Registered new tserver with Master: d137499136d9442b96caedae7d6902c8 (127.2.237.1:34371)
I20260620 23:03:48.099614 20150 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.1@KRBTEST.COM'} at 127.2.237.1:51315
WARNING: no policy specified for kudu/127.2.237.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
I20260620 23:03:48.133482 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:0
--local_ip_for_outbound_sockets=127.2.237.2
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:38561
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:40939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:48.478315 20345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:48.478683 20345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:48.478802 20345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:48.486081 20345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:48.486186 20345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:48.486302 20345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 23:03:48.496619 20345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:40939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:38561
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:03:48.497898 20345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:48.499182 20345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:48.512360 20350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:48.512647 20351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:48.513002 20353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:48.513882 20345 server_base.cc:1034] running on GCE node
I20260620 23:03:48.514437 20345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:48.515456 20345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:48.516654 20345 hybrid_clock.cc:648] HybridClock initialized: now 1781996628516604 us; error 39 us; skew 500 ppm
I20260620 23:03:48.516985 20345 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:03:48 dist-test-slave-f63b krb5kdc[20105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996628, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:48.520257 20345 init.cc:377] Logged in from keytab as kudu/127.2.237.2@KRBTEST.COM (short username kudu)
I20260620 23:03:48.521669 20345 webserver.cc:460] Webserver started at http://127.2.237.2:46327/ using document root <none> and password file <none>
I20260620 23:03:48.522377 20345 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:48.522507 20345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:48.522845 20345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:48.525738 20345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-1/data/instance:
uuid: "31e82321ca724e30a7be81dfc8a33072"
format_stamp: "Formatted at 2026-06-20 23:03:48 on dist-test-slave-f63b"
server_key: "9ef7fe0767ccddd8ab385196d774cdf7"
server_key_iv: "58f69a8631e80f936c5a1d67d4c48338"
server_key_version: "encryptionkey@0"
I20260620 23:03:48.526664 20345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance:
uuid: "31e82321ca724e30a7be81dfc8a33072"
format_stamp: "Formatted at 2026-06-20 23:03:48 on dist-test-slave-f63b"
server_key: "9ef7fe0767ccddd8ab385196d774cdf7"
server_key_iv: "58f69a8631e80f936c5a1d67d4c48338"
server_key_version: "encryptionkey@0"
I20260620 23:03:48.531920 20345 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260620 23:03:48.535444 20360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:48.536782 20345 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 23:03:48.536954 20345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "31e82321ca724e30a7be81dfc8a33072"
format_stamp: "Formatted at 2026-06-20 23:03:48 on dist-test-slave-f63b"
server_key: "9ef7fe0767ccddd8ab385196d774cdf7"
server_key_iv: "58f69a8631e80f936c5a1d67d4c48338"
server_key_version: "encryptionkey@0"
I20260620 23:03:48.537185 20345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:48.552287 20345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:48.556799 20345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:48.557143 20345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:48.558138 20345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:48.559520 20345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:48.559597 20345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:48.559688 20345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:48.559741 20345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:48.596277 20345 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:44027
I20260620 23:03:48.596321 20472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:44027 every 8 connection(s)
I20260620 23:03:48.597735 20345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 23:03:48.598194 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 20345
I20260620 23:03:48.598393 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance
I20260620 23:03:48.599169 2996 external_mini_cluster.cc:1428] Setting key b4ddd42d4de6f7f281127bbcfd5ee7dd
Jun 20 23:03:48 dist-test-slave-f63b krb5kdc[20105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996628, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:03:48.620985 20475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:48.600747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:49271 (local address 127.2.237.62:38561)
0620 23:03:48.601287 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:48.601293 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:48.602718 (+ 1425us) server_negotiation.cc:407] Connection header received
0620 23:03:48.605093 (+ 2375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:48.605101 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:48.605203 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:48.605327 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:48.607359 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:48.608618 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:48.610448 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:48.611005 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:48.611163 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:48.614915 (+ 3752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:48.614920 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:48.614923 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:48.614958 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:48.617702 (+ 2744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:48.618524 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:48.618531 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:48.618534 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:48.618738 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:48.619187 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:48.619192 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:48.619196 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:48.619485 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:48.619647 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:48.620363 (+ 716us) server_negotiation.cc:299] Negotiation successful
0620 23:03:48.620607 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":248,"threads_started":1}
I20260620 23:03:48.622052 20476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:48.601547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:38561 (local address 127.2.237.2:49271)
0620 23:03:48.602513 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:48.602564 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:48.604805 (+ 2241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:48.605601 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:48.605611 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:48.606267 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:48.607152 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:48.607190 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:48.608803 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:48.608808 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:48.610268 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:48.610281 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:48.611261 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:48.611271 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:48.611551 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:48.612305 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:48.612329 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:48.614666 (+ 2337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:48.617899 (+ 3233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:48.617908 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:48.617926 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:48.618387 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:48.618856 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:48.618861 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:48.618865 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:48.619067 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:48.619617 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:48.619625 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:48.619989 (+ 364us) client_negotiation.cc:769] Sending connection context
0620 23:03:48.620265 (+ 276us) client_negotiation.cc:240] Negotiation successful
0620 23:03:48.620842 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":402,"threads_started":1}
WARNING: no policy specified for kudu/127.2.237.3@KRBTEST.COM; defaulting to no policy
I20260620 23:03:48.623585 20473 heartbeater.cc:347] Connected to a master server at 127.2.237.62:38561
I20260620 23:03:48.623926 20473 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:48.624696 20473 heartbeater.cc:511] Master 127.2.237.62:38561 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.3@KRBTEST.COM" created.
I20260620 23:03:48.626235 20150 ts_manager.cc:194] Registered new tserver with Master: 31e82321ca724e30a7be81dfc8a33072 (127.2.237.2:44027)
I20260620 23:03:48.627180 20150 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.2@KRBTEST.COM'} at 127.2.237.2:49271
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
I20260620 23:03:48.658233 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:0
--local_ip_for_outbound_sockets=127.2.237.3
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:38561
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:40939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:48.994127 20479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:48.994458 20479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:48.994689 20479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:49.002066 20479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:49.002223 20479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:49.002393 20479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 23:03:49.012838 20479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:40939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:38561
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:03:49.014190 20479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:49.015695 20479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:49.028628 20484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:49.028729 20485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:49.028782 20487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:49.029920 20479 server_base.cc:1034] running on GCE node
I20260620 23:03:49.030632 20479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:49.031606 20479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:49.032827 20479 hybrid_clock.cc:648] HybridClock initialized: now 1781996629032779 us; error 42 us; skew 500 ppm
I20260620 23:03:49.033144 20479 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:03:49 dist-test-slave-f63b krb5kdc[20105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996629, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:49.036355 20479 init.cc:377] Logged in from keytab as kudu/127.2.237.3@KRBTEST.COM (short username kudu)
I20260620 23:03:49.037883 20479 webserver.cc:460] Webserver started at http://127.2.237.3:46693/ using document root <none> and password file <none>
I20260620 23:03:49.038616 20479 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:49.038762 20479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:49.039068 20479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:49.042711 20479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-2/data/instance:
uuid: "93c7b130a979449c848415ad81984ff9"
format_stamp: "Formatted at 2026-06-20 23:03:49 on dist-test-slave-f63b"
server_key: "2a78f00bdf1ccc8b88b397251c5eaa7a"
server_key_iv: "d6ab35a00024bb78dd03689108f04a28"
server_key_version: "encryptionkey@0"
I20260620 23:03:49.043501 20479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance:
uuid: "93c7b130a979449c848415ad81984ff9"
format_stamp: "Formatted at 2026-06-20 23:03:49 on dist-test-slave-f63b"
server_key: "2a78f00bdf1ccc8b88b397251c5eaa7a"
server_key_iv: "d6ab35a00024bb78dd03689108f04a28"
server_key_version: "encryptionkey@0"
I20260620 23:03:49.048647 20479 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260620 23:03:49.052328 20494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:49.053663 20479 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 23:03:49.053838 20479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "93c7b130a979449c848415ad81984ff9"
format_stamp: "Formatted at 2026-06-20 23:03:49 on dist-test-slave-f63b"
server_key: "2a78f00bdf1ccc8b88b397251c5eaa7a"
server_key_iv: "d6ab35a00024bb78dd03689108f04a28"
server_key_version: "encryptionkey@0"
I20260620 23:03:49.054090 20479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:49.075923 20479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:49.080140 20479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:49.080375 20479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:49.081279 20479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:49.082672 20479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:49.082777 20479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:49.082930 20479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:49.083002 20479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:49.102533 20339 heartbeater.cc:503] Master 127.2.237.62:38561 was elected leader, sending a full tablet report...
I20260620 23:03:49.116974 20479 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:33883
I20260620 23:03:49.117031 20606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:33883 every 8 connection(s)
I20260620 23:03:49.118651 20479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 23:03:49.121990 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 20479
I20260620 23:03:49.122177 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance
I20260620 23:03:49.122809 2996 external_mini_cluster.cc:1428] Setting key 0052da21f536e6a1a299bd0f36748050
Jun 20 23:03:49 dist-test-slave-f63b krb5kdc[20105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996629, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:03:49.140879 20609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:49.121475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:51747 (local address 127.2.237.62:38561)
0620 23:03:49.122022 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:49.122030 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:49.123350 (+ 1320us) server_negotiation.cc:407] Connection header received
0620 23:03:49.125790 (+ 2440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:49.125796 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:49.125927 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:49.126104 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:49.127752 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.128648 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.130333 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.130826 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.130940 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:49.134707 (+ 3767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:49.134713 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:49.134716 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:49.134750 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:49.137807 (+ 3057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:49.138785 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:49.138789 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:49.138791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:49.138912 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:49.139302 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:49.139305 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:49.139307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:49.139591 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:49.139710 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:49.140390 (+ 680us) server_negotiation.cc:299] Negotiation successful
0620 23:03:49.140614 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":263,"threads_started":1}
I20260620 23:03:49.142010 20610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:49.122173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:38561 (local address 127.2.237.3:51747)
0620 23:03:49.123145 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:49.123204 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:49.125456 (+ 2252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:49.126315 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:49.126323 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:49.126945 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:49.127574 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:49.127592 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.128820 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.128825 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:49.130169 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:49.130183 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.131050 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.131056 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:49.131273 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:49.132047 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:49.132078 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:49.134387 (+ 2309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:49.137939 (+ 3552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:49.137949 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:49.137970 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:49.138666 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:49.139014 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:49.139017 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:49.139019 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:49.139200 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:49.139696 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:49.139702 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:49.140038 (+ 336us) client_negotiation.cc:769] Sending connection context
0620 23:03:49.140258 (+ 220us) client_negotiation.cc:240] Negotiation successful
0620 23:03:49.140790 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":354,"threads_started":1}
I20260620 23:03:49.143538 20607 heartbeater.cc:347] Connected to a master server at 127.2.237.62:38561
I20260620 23:03:49.143877 20607 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:49.144676 20607 heartbeater.cc:511] Master 127.2.237.62:38561 requested a full tablet report, sending...
I20260620 23:03:49.146219 20150 ts_manager.cc:194] Registered new tserver with Master: 93c7b130a979449c848415ad81984ff9 (127.2.237.3:33883)
I20260620 23:03:49.147156 20150 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.3@KRBTEST.COM'} at 127.2.237.3:51747
I20260620 23:03:49.151345 2996 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:03:49.169500 20609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:49.156117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49430 (local address 127.2.237.62:38561)
0620 23:03:49.156379 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:49.156386 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:49.156673 (+ 287us) server_negotiation.cc:407] Connection header received
0620 23:03:49.156806 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:49.156810 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:49.156871 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:49.156951 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:49.158739 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.159472 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.161083 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.161533 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.161645 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:49.164206 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:49.164211 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:49.164215 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:49.164247 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:49.166905 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:49.167566 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:49.167570 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:49.167572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:49.167727 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:49.168090 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:49.168093 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:49.168095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:49.168425 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:49.168632 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:49.168956 (+ 324us) server_negotiation.cc:299] Negotiation successful
0620 23:03:49.169128 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260620 23:03:49.173561 20150 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49430:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:03:49.180493 20150 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:03:49.213384 20621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:49.193350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34388 (local address 127.2.237.1:34371)
0620 23:03:49.203358 (+ 10008us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:49.203371 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:49.204205 (+ 834us) server_negotiation.cc:407] Connection header received
0620 23:03:49.204531 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:49.204541 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:49.204755 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:49.205003 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:49.206816 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.208460 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.210888 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.212207 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.212337 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:49.212580 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:49.212905 (+ 325us) server_negotiation.cc:299] Negotiation successful
0620 23:03:49.213065 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9779,"thread_start_us":9602,"threads_started":1}
I20260620 23:03:49.213419 20620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:49.193207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:34371 (local address 127.0.0.1:34388)
0620 23:03:49.203980 (+ 10773us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:49.204034 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:49.204298 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:49.205285 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:49.205290 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:49.205321 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:49.206459 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:49.206474 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.208645 (+ 2171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.208650 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:49.210678 (+ 2028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:49.210699 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.212410 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.212417 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:49.212613 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:49.212672 (+ 59us) client_negotiation.cc:769] Sending connection context
0620 23:03:49.212811 (+ 139us) client_negotiation.cc:240] Negotiation successful
0620 23:03:49.212995 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10505,"mutex_wait_us":45,"thread_start_us":10253,"threads_started":1}
I20260620 23:03:49.213622 20623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:49.194165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38260 (local address 127.2.237.2:44027)
0620 23:03:49.202801 (+ 8636us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:49.202813 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:49.204253 (+ 1440us) server_negotiation.cc:407] Connection header received
0620 23:03:49.204489 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:49.204499 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:49.204813 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:49.205260 (+ 447us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:49.206934 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.208331 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.210902 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.212267 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.212427 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:49.212619 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:49.213149 (+ 530us) server_negotiation.cc:299] Negotiation successful
0620 23:03:49.213317 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8468,"thread_start_us":8333,"threads_started":1}
I20260620 23:03:49.214366 20622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:49.193376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:44027 (local address 127.0.0.1:38260)
0620 23:03:49.203979 (+ 10603us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:49.204032 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:49.204211 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:49.205084 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:49.205095 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:49.205138 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:49.206760 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:49.206773 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.208465 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.208470 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:49.210711 (+ 2241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:49.210731 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.212763 (+ 2032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.212772 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:49.212945 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:49.212974 (+ 29us) client_negotiation.cc:769] Sending connection context
0620 23:03:49.213818 (+ 844us) client_negotiation.cc:240] Negotiation successful
0620 23:03:49.213971 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10391,"mutex_wait_us":64,"thread_start_us":10225,"threads_started":1}
I20260620 23:03:49.214772 20625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:49.204414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:33883 (local address 127.0.0.1:60400)
0620 23:03:49.206421 (+ 2007us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:49.206439 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:49.206563 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:49.207324 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:49.207328 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:49.207353 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:49.207672 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:49.207682 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.209580 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.209585 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:49.211876 (+ 2291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:49.211891 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.213851 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.213859 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:49.214058 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:49.214098 (+ 40us) client_negotiation.cc:769] Sending connection context
0620 23:03:49.214195 (+ 97us) client_negotiation.cc:240] Negotiation successful
0620 23:03:49.214320 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1861,"mutex_wait_us":2,"thread_start_us":1637,"threads_started":1}
I20260620 23:03:49.214931 20624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:49.204748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60400 (local address 127.2.237.3:33883)
0620 23:03:49.206025 (+ 1277us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:49.206036 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:49.206729 (+ 693us) server_negotiation.cc:407] Connection header received
0620 23:03:49.206851 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:49.206860 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:49.207116 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:49.207308 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:49.207886 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.209425 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.212048 (+ 2623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.213350 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.213480 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:49.213707 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:49.214255 (+ 548us) server_negotiation.cc:299] Negotiation successful
0620 23:03:49.214446 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1104,"thread_start_us":948,"threads_started":1}
I20260620 23:03:49.217955 20274 tablet_service.cc:1467] Processing CreateTablet for tablet da06f021c9134f3b9d0b7746c065fadd (DEFAULT_TABLE table=test-table [id=0cb5f31d85974ccfb066f7eb25efe557]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:49.218374 20542 tablet_service.cc:1467] Processing CreateTablet for tablet da06f021c9134f3b9d0b7746c065fadd (DEFAULT_TABLE table=test-table [id=0cb5f31d85974ccfb066f7eb25efe557]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:49.219333 20274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da06f021c9134f3b9d0b7746c065fadd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:49.219866 20542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da06f021c9134f3b9d0b7746c065fadd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:49.221973 20408 tablet_service.cc:1467] Processing CreateTablet for tablet da06f021c9134f3b9d0b7746c065fadd (DEFAULT_TABLE table=test-table [id=0cb5f31d85974ccfb066f7eb25efe557]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:49.223307 20408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da06f021c9134f3b9d0b7746c065fadd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:49.231144 20626 tablet_bootstrap.cc:492] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9: Bootstrap starting.
I20260620 23:03:49.233814 20627 tablet_bootstrap.cc:492] T da06f021c9134f3b9d0b7746c065fadd P d137499136d9442b96caedae7d6902c8: Bootstrap starting.
I20260620 23:03:49.234992 20626 tablet_bootstrap.cc:654] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:49.236306 20626 log.cc:824] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:49.237380 20627 tablet_bootstrap.cc:654] T da06f021c9134f3b9d0b7746c065fadd P d137499136d9442b96caedae7d6902c8: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:49.238703 20627 log.cc:824] T da06f021c9134f3b9d0b7746c065fadd P d137499136d9442b96caedae7d6902c8: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:49.239610 20626 tablet_bootstrap.cc:492] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9: No bootstrap required, opened a new log
I20260620 23:03:49.239888 20626 ts_tablet_manager.cc:1397] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260620 23:03:49.241972 20627 tablet_bootstrap.cc:492] T da06f021c9134f3b9d0b7746c065fadd P d137499136d9442b96caedae7d6902c8: No bootstrap required, opened a new log
I20260620 23:03:49.242249 20627 ts_tablet_manager.cc:1397] T da06f021c9134f3b9d0b7746c065fadd P d137499136d9442b96caedae7d6902c8: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260620 23:03:49.243309 20629 tablet_bootstrap.cc:492] T da06f021c9134f3b9d0b7746c065fadd P 31e82321ca724e30a7be81dfc8a33072: Bootstrap starting.
I20260620 23:03:49.247134 20629 tablet_bootstrap.cc:654] T da06f021c9134f3b9d0b7746c065fadd P 31e82321ca724e30a7be81dfc8a33072: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:49.248221 20629 log.cc:824] T da06f021c9134f3b9d0b7746c065fadd P 31e82321ca724e30a7be81dfc8a33072: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:49.251025 20629 tablet_bootstrap.cc:492] T da06f021c9134f3b9d0b7746c065fadd P 31e82321ca724e30a7be81dfc8a33072: No bootstrap required, opened a new log
I20260620 23:03:49.250844 20626 raft_consensus.cc:348] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d137499136d9442b96caedae7d6902c8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 34371 } } peers { permanent_uuid: "31e82321ca724e30a7be81dfc8a33072" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 44027 } } peers { permanent_uuid: "93c7b130a979449c848415ad81984ff9" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 33883 } }
I20260620 23:03:49.251312 20626 raft_consensus.cc:374] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:49.251348 20629 ts_tablet_manager.cc:1397] T da06f021c9134f3b9d0b7746c065fadd P 31e82321ca724e30a7be81dfc8a33072: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260620 23:03:49.251415 20626 raft_consensus.cc:729] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93c7b130a979449c848415ad81984ff9, State: Initialized, Role: FOLLOWER
I20260620 23:03:49.252075 20626 consensus_queue.cc:260] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d137499136d9442b96caedae7d6902c8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 34371 } } peers { permanent_uuid: "31e82321ca724e30a7be81dfc8a33072" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 44027 } } peers { permanent_uuid: "93c7b130a979449c848415ad81984ff9" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 33883 } }
I20260620 23:03:49.253206 20627 raft_consensus.cc:348] T da06f021c9134f3b9d0b7746c065fadd P d137499136d9442b96caedae7d6902c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d137499136d9442b96caedae7d6902c8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 34371 } } peers { permanent_uuid: "31e82321ca724e30a7be81dfc8a33072" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 44027 } } peers { permanent_uuid: "93c7b130a979449c848415ad81984ff9" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 33883 } }
I20260620 23:03:49.253749 20627 raft_consensus.cc:374] T da06f021c9134f3b9d0b7746c065fadd P d137499136d9442b96caedae7d6902c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:49.253872 20627 raft_consensus.cc:729] T da06f021c9134f3b9d0b7746c065fadd P d137499136d9442b96caedae7d6902c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d137499136d9442b96caedae7d6902c8, State: Initialized, Role: FOLLOWER
I20260620 23:03:49.254592 20627 consensus_queue.cc:260] T da06f021c9134f3b9d0b7746c065fadd P d137499136d9442b96caedae7d6902c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d137499136d9442b96caedae7d6902c8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 34371 } } peers { permanent_uuid: "31e82321ca724e30a7be81dfc8a33072" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 44027 } } peers { permanent_uuid: "93c7b130a979449c848415ad81984ff9" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 33883 } }
I20260620 23:03:49.255362 20607 heartbeater.cc:503] Master 127.2.237.62:38561 was elected leader, sending a full tablet report...
I20260620 23:03:49.255986 20626 ts_tablet_manager.cc:1428] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9: Time spent starting tablet: real 0.016s user 0.013s sys 0.003s
I20260620 23:03:49.260277 20629 raft_consensus.cc:348] T da06f021c9134f3b9d0b7746c065fadd P 31e82321ca724e30a7be81dfc8a33072 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d137499136d9442b96caedae7d6902c8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 34371 } } peers { permanent_uuid: "31e82321ca724e30a7be81dfc8a33072" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 44027 } } peers { permanent_uuid: "93c7b130a979449c848415ad81984ff9" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 33883 } }
I20260620 23:03:49.260643 20629 raft_consensus.cc:374] T da06f021c9134f3b9d0b7746c065fadd P 31e82321ca724e30a7be81dfc8a33072 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:49.260767 20629 raft_consensus.cc:729] T da06f021c9134f3b9d0b7746c065fadd P 31e82321ca724e30a7be81dfc8a33072 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31e82321ca724e30a7be81dfc8a33072, State: Initialized, Role: FOLLOWER
I20260620 23:03:49.261345 20629 consensus_queue.cc:260] T da06f021c9134f3b9d0b7746c065fadd P 31e82321ca724e30a7be81dfc8a33072 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d137499136d9442b96caedae7d6902c8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 34371 } } peers { permanent_uuid: "31e82321ca724e30a7be81dfc8a33072" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 44027 } } peers { permanent_uuid: "93c7b130a979449c848415ad81984ff9" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 33883 } }
I20260620 23:03:49.262620 20473 heartbeater.cc:503] Master 127.2.237.62:38561 was elected leader, sending a full tablet report...
I20260620 23:03:49.263259 20629 ts_tablet_manager.cc:1428] T da06f021c9134f3b9d0b7746c065fadd P 31e82321ca724e30a7be81dfc8a33072: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260620 23:03:49.263842 20627 ts_tablet_manager.cc:1428] T da06f021c9134f3b9d0b7746c065fadd P d137499136d9442b96caedae7d6902c8: Time spent starting tablet: real 0.021s user 0.016s sys 0.005s
W20260620 23:03:49.324795 20340 tablet.cc:2369] T da06f021c9134f3b9d0b7746c065fadd P d137499136d9442b96caedae7d6902c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:03:49.349956 20474 tablet.cc:2369] T da06f021c9134f3b9d0b7746c065fadd P 31e82321ca724e30a7be81dfc8a33072: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260620 23:03:49.370561 20608 tablet.cc:2369] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:03:49.440562 20632 raft_consensus.cc:482] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:03:49.440842 20632 raft_consensus.cc:504] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d137499136d9442b96caedae7d6902c8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 34371 } } peers { permanent_uuid: "31e82321ca724e30a7be81dfc8a33072" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 44027 } } peers { permanent_uuid: "93c7b130a979449c848415ad81984ff9" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 33883 } }
I20260620 23:03:49.442874 20632 leader_election.cc:288] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d137499136d9442b96caedae7d6902c8 (127.2.237.1:34371), 31e82321ca724e30a7be81dfc8a33072 (127.2.237.2:44027)
I20260620 23:03:49.449551 20621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:49.443307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:58545 (local address 127.2.237.1:34371)
0620 23:03:49.443545 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:49.443551 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:49.443731 (+ 180us) server_negotiation.cc:407] Connection header received
0620 23:03:49.443948 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:49.443952 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:49.444016 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:49.444122 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:49.444538 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.445539 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.447396 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.448646 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.448748 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:49.448823 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:49.449171 (+ 348us) server_negotiation.cc:299] Negotiation successful
0620 23:03:49.449330 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260620 23:03:49.449642 20610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:49.443269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:34371 (local address 127.2.237.3:58545)
0620 23:03:49.443615 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:49.443636 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:49.443782 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:49.444142 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:49.444146 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:49.444172 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:49.444379 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:49.444393 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.445663 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.445669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:49.447238 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:49.447250 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.448796 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.448802 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:49.448996 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:49.449043 (+ 47us) client_negotiation.cc:769] Sending connection context
0620 23:03:49.449140 (+ 97us) client_negotiation.cc:240] Negotiation successful
0620 23:03:49.449288 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":4}
I20260620 23:03:49.450786 20623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:49.443398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:56391 (local address 127.2.237.2:44027)
0620 23:03:49.443628 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:49.443636 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:49.444332 (+ 696us) server_negotiation.cc:407] Connection header received
0620 23:03:49.444455 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:49.444463 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:49.444585 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:49.444653 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:49.445259 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.446192 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.448528 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.449654 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.449789 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:49.449899 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:49.450381 (+ 482us) server_negotiation.cc:299] Negotiation successful
0620 23:03:49.450514 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260620 23:03:49.450855 20635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:49.443392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:44027 (local address 127.2.237.3:56391)
0620 23:03:49.444231 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:49.444245 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:49.444350 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:49.444701 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:49.444705 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:49.444729 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:49.445100 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:49.445112 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.446366 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.446370 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:49.448361 (+ 1991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:49.448376 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.450025 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.450030 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:49.450183 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:49.450204 (+ 21us) client_negotiation.cc:769] Sending connection context
0620 23:03:49.450327 (+ 123us) client_negotiation.cc:240] Negotiation successful
0620 23:03:49.450460 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"mutex_wait_us":43,"thread_start_us":454,"threads_started":1}
I20260620 23:03:49.450861 20294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da06f021c9134f3b9d0b7746c065fadd" candidate_uuid: "93c7b130a979449c848415ad81984ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d137499136d9442b96caedae7d6902c8" is_pre_election: true
I20260620 23:03:49.451442 20294 raft_consensus.cc:2454] T da06f021c9134f3b9d0b7746c065fadd P d137499136d9442b96caedae7d6902c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93c7b130a979449c848415ad81984ff9 in term 0.
I20260620 23:03:49.451900 20428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da06f021c9134f3b9d0b7746c065fadd" candidate_uuid: "93c7b130a979449c848415ad81984ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31e82321ca724e30a7be81dfc8a33072" is_pre_election: true
I20260620 23:03:49.452227 20496 leader_election.cc:302] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93c7b130a979449c848415ad81984ff9, d137499136d9442b96caedae7d6902c8; no voters:
I20260620 23:03:49.452399 20428 raft_consensus.cc:2454] T da06f021c9134f3b9d0b7746c065fadd P 31e82321ca724e30a7be81dfc8a33072 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93c7b130a979449c848415ad81984ff9 in term 0.
I20260620 23:03:49.452682 20632 raft_consensus.cc:2781] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:03:49.452886 20632 raft_consensus.cc:482] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:03:49.453020 20632 raft_consensus.cc:3037] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:49.455446 20632 raft_consensus.cc:504] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d137499136d9442b96caedae7d6902c8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 34371 } } peers { permanent_uuid: "31e82321ca724e30a7be81dfc8a33072" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 44027 } } peers { permanent_uuid: "93c7b130a979449c848415ad81984ff9" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 33883 } }
I20260620 23:03:49.456086 20632 leader_election.cc:288] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9 [CANDIDATE]: Term 1 election: Requested vote from peers d137499136d9442b96caedae7d6902c8 (127.2.237.1:34371), 31e82321ca724e30a7be81dfc8a33072 (127.2.237.2:44027)
I20260620 23:03:49.456456 20294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da06f021c9134f3b9d0b7746c065fadd" candidate_uuid: "93c7b130a979449c848415ad81984ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d137499136d9442b96caedae7d6902c8"
I20260620 23:03:49.456555 20428 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da06f021c9134f3b9d0b7746c065fadd" candidate_uuid: "93c7b130a979449c848415ad81984ff9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31e82321ca724e30a7be81dfc8a33072"
I20260620 23:03:49.456660 20294 raft_consensus.cc:3037] T da06f021c9134f3b9d0b7746c065fadd P d137499136d9442b96caedae7d6902c8 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:49.456768 20428 raft_consensus.cc:3037] T da06f021c9134f3b9d0b7746c065fadd P 31e82321ca724e30a7be81dfc8a33072 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:49.459177 20294 raft_consensus.cc:2454] T da06f021c9134f3b9d0b7746c065fadd P d137499136d9442b96caedae7d6902c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93c7b130a979449c848415ad81984ff9 in term 1.
I20260620 23:03:49.459556 20496 leader_election.cc:302] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93c7b130a979449c848415ad81984ff9, d137499136d9442b96caedae7d6902c8; no voters:
I20260620 23:03:49.459860 20632 raft_consensus.cc:2781] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:49.459913 20428 raft_consensus.cc:2454] T da06f021c9134f3b9d0b7746c065fadd P 31e82321ca724e30a7be81dfc8a33072 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93c7b130a979449c848415ad81984ff9 in term 1.
I20260620 23:03:49.460386 20632 raft_consensus.cc:686] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9 [term 1 LEADER]: Becoming Leader. State: Replica: 93c7b130a979449c848415ad81984ff9, State: Running, Role: LEADER
I20260620 23:03:49.460886 20632 consensus_queue.cc:237] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d137499136d9442b96caedae7d6902c8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 34371 } } peers { permanent_uuid: "31e82321ca724e30a7be81dfc8a33072" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 44027 } } peers { permanent_uuid: "93c7b130a979449c848415ad81984ff9" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 33883 } }
I20260620 23:03:49.465363 20148 catalog_manager.cc:5495] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 93c7b130a979449c848415ad81984ff9 (127.2.237.3). New cstate: current_term: 1 leader_uuid: "93c7b130a979449c848415ad81984ff9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d137499136d9442b96caedae7d6902c8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 34371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31e82321ca724e30a7be81dfc8a33072" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 44027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93c7b130a979449c848415ad81984ff9" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 33883 } health_report { overall_health: HEALTHY } } }
I20260620 23:03:49.502636 20624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:49.496633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60404 (local address 127.2.237.3:33883)
0620 23:03:49.496850 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:49.496854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:49.496875 (+ 21us) server_negotiation.cc:407] Connection header received
0620 23:03:49.497093 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:49.497098 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:49.497195 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:49.497330 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:03:49.497682 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.498505 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.500139 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.500774 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.500889 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:49.501453 (+ 564us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:03:49.501999 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:03:49.502120 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:49.502273 (+ 153us) server_negotiation.cc:299] Negotiation successful
0620 23:03:49.502349 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260620 23:03:49.511361 20428 raft_consensus.cc:1261] T da06f021c9134f3b9d0b7746c065fadd P 31e82321ca724e30a7be81dfc8a33072 [term 1 FOLLOWER]: Refusing update from remote peer 93c7b130a979449c848415ad81984ff9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:49.511386 20294 raft_consensus.cc:1261] T da06f021c9134f3b9d0b7746c065fadd P d137499136d9442b96caedae7d6902c8 [term 1 FOLLOWER]: Refusing update from remote peer 93c7b130a979449c848415ad81984ff9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:49.512364 20636 consensus_queue.cc:1035] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d137499136d9442b96caedae7d6902c8" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 34371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:03:49.512776 20632 consensus_queue.cc:1035] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31e82321ca724e30a7be81dfc8a33072" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 44027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:03:49.540095 20150 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:03:49.540463 20150 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:03:49.544575 20150 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 759fd70d56f74ccba53bdd0ff8964755: Sending DeleteTablet for 3 replicas of tablet da06f021c9134f3b9d0b7746c065fadd
I20260620 23:03:49.545840 20408 tablet_service.cc:1514] Processing DeleteTablet for tablet da06f021c9134f3b9d0b7746c065fadd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:49 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:38260
I20260620 23:03:49.546046 20274 tablet_service.cc:1514] Processing DeleteTablet for tablet da06f021c9134f3b9d0b7746c065fadd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:49 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:34388
I20260620 23:03:49.546166 20542 tablet_service.cc:1514] Processing DeleteTablet for tablet da06f021c9134f3b9d0b7746c065fadd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:49 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:60400
I20260620 23:03:49.548082 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 20211
I20260620 23:03:49.562034 20650 tablet_replica.cc:330] T da06f021c9134f3b9d0b7746c065fadd P d137499136d9442b96caedae7d6902c8: stopping tablet replica
I20260620 23:03:49.562610 20650 raft_consensus.cc:2229] T da06f021c9134f3b9d0b7746c065fadd P d137499136d9442b96caedae7d6902c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:03:49.563201 20650 raft_consensus.cc:2258] T da06f021c9134f3b9d0b7746c065fadd P d137499136d9442b96caedae7d6902c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:49.565604 20650 ts_tablet_manager.cc:1905] T da06f021c9134f3b9d0b7746c065fadd P d137499136d9442b96caedae7d6902c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:49.572959 20650 ts_tablet_manager.cc:1918] T da06f021c9134f3b9d0b7746c065fadd P d137499136d9442b96caedae7d6902c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:03:49.573141 20650 log.cc:1196] T da06f021c9134f3b9d0b7746c065fadd P d137499136d9442b96caedae7d6902c8: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-0/wal/wals/da06f021c9134f3b9d0b7746c065fadd
I20260620 23:03:49.573688 20650 ts_tablet_manager.cc:1939] T da06f021c9134f3b9d0b7746c065fadd P d137499136d9442b96caedae7d6902c8: Deleting consensus metadata
I20260620 23:03:49.575717 20135 catalog_manager.cc:4841] TS d137499136d9442b96caedae7d6902c8 (127.2.237.1:34371): tablet da06f021c9134f3b9d0b7746c065fadd (table test-table [id=0cb5f31d85974ccfb066f7eb25efe557]) successfully deleted
I20260620 23:03:49.578047 20649 tablet_replica.cc:330] T da06f021c9134f3b9d0b7746c065fadd P 31e82321ca724e30a7be81dfc8a33072: stopping tablet replica
I20260620 23:03:49.578750 20649 raft_consensus.cc:2229] T da06f021c9134f3b9d0b7746c065fadd P 31e82321ca724e30a7be81dfc8a33072 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:03:49.579550 20649 raft_consensus.cc:2258] T da06f021c9134f3b9d0b7746c065fadd P 31e82321ca724e30a7be81dfc8a33072 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:49.582454 20649 ts_tablet_manager.cc:1905] T da06f021c9134f3b9d0b7746c065fadd P 31e82321ca724e30a7be81dfc8a33072: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:49.582966 20651 tablet_replica.cc:330] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9: stopping tablet replica
I20260620 23:03:49.583400 20651 raft_consensus.cc:2229] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9 [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:03:49.584067 20651 raft_consensus.cc:2258] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:49.586263 20651 ts_tablet_manager.cc:1905] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9: Deleting tablet data with delete state TABLET_DATA_DELETED
Jun 20 23:03:49 dist-test-slave-f63b krb5kdc[20105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.1@KRBTEST.COM
I20260620 23:03:49.589150 20649 ts_tablet_manager.cc:1918] T da06f021c9134f3b9d0b7746c065fadd P 31e82321ca724e30a7be81dfc8a33072: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:03:49.589310 20649 log.cc:1196] T da06f021c9134f3b9d0b7746c065fadd P 31e82321ca724e30a7be81dfc8a33072: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-1/wal/wals/da06f021c9134f3b9d0b7746c065fadd
I20260620 23:03:49.589838 20649 ts_tablet_manager.cc:1939] T da06f021c9134f3b9d0b7746c065fadd P 31e82321ca724e30a7be81dfc8a33072: Deleting consensus metadata
I20260620 23:03:49.591930 20136 catalog_manager.cc:4841] TS 31e82321ca724e30a7be81dfc8a33072 (127.2.237.2:44027): tablet da06f021c9134f3b9d0b7746c065fadd (table test-table [id=0cb5f31d85974ccfb066f7eb25efe557]) successfully deleted
I20260620 23:03:49.593670 20651 ts_tablet_manager.cc:1918] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:03:49.593904 20651 log.cc:1196] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781996318807807-2996-0/minicluster-data/ts-2/wal/wals/da06f021c9134f3b9d0b7746c065fadd
I20260620 23:03:49.594381 20651 ts_tablet_manager.cc:1939] T da06f021c9134f3b9d0b7746c065fadd P 93c7b130a979449c848415ad81984ff9: Deleting consensus metadata
I20260620 23:03:49.595954 20135 catalog_manager.cc:4841] TS 93c7b130a979449c848415ad81984ff9 (127.2.237.3:33883): tablet da06f021c9134f3b9d0b7746c065fadd (table test-table [id=0cb5f31d85974ccfb066f7eb25efe557]) successfully deleted
I20260620 23:03:49.597523 20621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:49.549842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34402 (local address 127.2.237.1:34371)
0620 23:03:49.550092 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:49.550101 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:49.574346 (+ 24245us) server_negotiation.cc:407] Connection header received
0620 23:03:49.574698 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:49.574707 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:49.574830 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:49.574926 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:49.577457 (+ 2531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.578607 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.580919 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.581552 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.581651 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:49.588937 (+ 7286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:49.588953 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:49.588972 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:49.589032 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:49.593603 (+ 4571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:49.594622 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:49.594633 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:49.594637 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:49.594811 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:49.595377 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:49.595385 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:49.595390 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:49.595925 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:49.596230 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:49.596903 (+ 673us) server_negotiation.cc:299] Negotiation successful
0620 23:03:49.597139 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260620 23:03:49.599445 20334 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:03:49.697639 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 20211
I20260620 23:03:49.721493 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 20345
Jun 20 23:03:49 dist-test-slave-f63b krb5kdc[20105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.2@KRBTEST.COM
I20260620 23:03:49.737406 20623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:49.722361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38276 (local address 127.2.237.2:44027)
0620 23:03:49.722559 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:49.722566 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:49.722651 (+ 85us) server_negotiation.cc:407] Connection header received
0620 23:03:49.722723 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:49.722728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:49.722819 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:49.722929 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:49.724180 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.724940 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.726224 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.726705 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.726837 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:49.731596 (+ 4759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:49.731610 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:49.731628 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:49.731663 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:49.734509 (+ 2846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:49.735331 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:49.735346 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:49.735349 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:49.735474 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:49.735841 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:49.735846 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:49.735849 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:49.736235 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:49.736552 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:49.736844 (+ 292us) server_negotiation.cc:299] Negotiation successful
0620 23:03:49.737062 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260620 23:03:49.738229 20468 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:03:49.831737 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 20345
I20260620 23:03:49.854805 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 20479
Jun 20 23:03:49 dist-test-slave-f63b krb5kdc[20105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.3@KRBTEST.COM
I20260620 23:03:49.873857 20624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:49.855888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60420 (local address 127.2.237.3:33883)
0620 23:03:49.856073 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:49.856078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:49.856103 (+ 25us) server_negotiation.cc:407] Connection header received
0620 23:03:49.856218 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:49.856222 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:49.856290 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:49.856398 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:49.858236 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.859317 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.861198 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:49.861684 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:49.861776 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:49.867669 (+ 5893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:49.867695 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:49.867711 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:49.867746 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:49.870872 (+ 3126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:49.871586 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:49.871593 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:49.871596 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:49.871823 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:49.872215 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:49.872221 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:49.872224 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:49.872604 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:49.872900 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:49.873224 (+ 324us) server_negotiation.cc:299] Negotiation successful
0620 23:03:49.873455 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260620 23:03:49.874517 20602 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:03:49.999598 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 20479
I20260620 23:03:50.019683 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 20118
I20260620 23:03:50.020905 20180 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:03:50.131943 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 20118
2026-06-20T23:03:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4482 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-20T23:03:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:03:50Z Disabled control of system clock
I20260620 23:03:50.210084 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:37103
--webserver_interface=127.2.237.62
--webserver_port=0
--builtin_ntp_servers=127.2.237.20:39295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:37103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 23:03:50.541057 20660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:50.541391 20660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:50.541513 20660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:50.548635 20660 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:03:50.548743 20660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:50.548780 20660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:50.548833 20660 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:03:50.548878 20660 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:03:50.558009 20660 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:39295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:37103
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:37103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:03:50.559270 20660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:50.561002 20660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:50.572585 20665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:50.572682 20668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:50.572659 20666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:50.573669 20660 server_base.cc:1034] running on GCE node
I20260620 23:03:50.574472 20660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:50.575906 20660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:50.577154 20660 hybrid_clock.cc:648] HybridClock initialized: now 1781996630577093 us; error 57 us; skew 500 ppm
I20260620 23:03:50.577538 20660 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 23:03:50.580015 20660 webserver.cc:460] Webserver started at http://127.2.237.62:34645/ using document root <none> and password file <none>
I20260620 23:03:50.580713 20660 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:50.580844 20660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:50.581174 20660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:50.584133 20660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "229aee15bd3a41cabddf2b06290fe091"
format_stamp: "Formatted at 2026-06-20 23:03:50 on dist-test-slave-f63b"
server_key: "084559291f9e674aba06459cde97f56a"
server_key_iv: "f805e607ea39664344ca76f07cdeeae2"
server_key_version: "encryptionkey@0"
I20260620 23:03:50.584949 20660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "229aee15bd3a41cabddf2b06290fe091"
format_stamp: "Formatted at 2026-06-20 23:03:50 on dist-test-slave-f63b"
server_key: "084559291f9e674aba06459cde97f56a"
server_key_iv: "f805e607ea39664344ca76f07cdeeae2"
server_key_version: "encryptionkey@0"
I20260620 23:03:50.590170 20660 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260620 23:03:50.593782 20674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:50.595142 20660 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 23:03:50.595335 20660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "229aee15bd3a41cabddf2b06290fe091"
format_stamp: "Formatted at 2026-06-20 23:03:50 on dist-test-slave-f63b"
server_key: "084559291f9e674aba06459cde97f56a"
server_key_iv: "f805e607ea39664344ca76f07cdeeae2"
server_key_version: "encryptionkey@0"
I20260620 23:03:50.595566 20660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:50.612443 20660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:50.613358 20660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:50.613659 20660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:50.632491 20660 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:37103
I20260620 23:03:50.632524 20725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:37103 every 8 connection(s)
I20260620 23:03:50.633857 20660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:03:50.637730 20726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:50.643561 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 20660
I20260620 23:03:50.643735 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:03:50.644343 2996 external_mini_cluster.cc:1428] Setting key 226f730335b44d60902c6fb6f4bddf40
I20260620 23:03:50.649472 20726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 229aee15bd3a41cabddf2b06290fe091: Bootstrap starting.
I20260620 23:03:50.652686 20726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 229aee15bd3a41cabddf2b06290fe091: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:50.653689 20726 log.cc:824] T 00000000000000000000000000000000 P 229aee15bd3a41cabddf2b06290fe091: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:50.656894 20726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 229aee15bd3a41cabddf2b06290fe091: No bootstrap required, opened a new log
I20260620 23:03:50.658237 20729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:50.646424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57526 (local address 127.2.237.62:37103)
0620 23:03:50.647513 (+ 1089us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:50.647524 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:50.647572 (+ 48us) server_negotiation.cc:407] Connection header received
0620 23:03:50.649277 (+ 1705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:50.649301 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:50.649814 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:50.650213 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:50.650801 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:50.652080 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:50.653554 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:50.654460 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:50.654689 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:50.655818 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:50.655825 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:50.655839 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:50.656404 (+ 565us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:50.656490 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:50.656511 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:50.656595 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:50.656833 (+ 238us) server_negotiation.cc:299] Negotiation successful
0620 23:03:50.657205 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":341,"threads_started":1}
I20260620 23:03:50.664861 20726 raft_consensus.cc:348] T 00000000000000000000000000000000 P 229aee15bd3a41cabddf2b06290fe091 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "229aee15bd3a41cabddf2b06290fe091" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37103 } }
I20260620 23:03:50.665232 20726 raft_consensus.cc:374] T 00000000000000000000000000000000 P 229aee15bd3a41cabddf2b06290fe091 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:50.665341 20726 raft_consensus.cc:729] T 00000000000000000000000000000000 P 229aee15bd3a41cabddf2b06290fe091 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 229aee15bd3a41cabddf2b06290fe091, State: Initialized, Role: FOLLOWER
I20260620 23:03:50.665967 20726 consensus_queue.cc:260] T 00000000000000000000000000000000 P 229aee15bd3a41cabddf2b06290fe091 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "229aee15bd3a41cabddf2b06290fe091" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37103 } }
I20260620 23:03:50.666240 20726 raft_consensus.cc:388] T 00000000000000000000000000000000 P 229aee15bd3a41cabddf2b06290fe091 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:03:50.666402 20726 raft_consensus.cc:482] T 00000000000000000000000000000000 P 229aee15bd3a41cabddf2b06290fe091 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:03:50.666653 20726 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 229aee15bd3a41cabddf2b06290fe091 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:50.668880 20726 raft_consensus.cc:504] T 00000000000000000000000000000000 P 229aee15bd3a41cabddf2b06290fe091 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "229aee15bd3a41cabddf2b06290fe091" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37103 } }
I20260620 23:03:50.669495 20726 leader_election.cc:302] T 00000000000000000000000000000000 P 229aee15bd3a41cabddf2b06290fe091 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 229aee15bd3a41cabddf2b06290fe091; no voters:
I20260620 23:03:50.670110 20726 leader_election.cc:288] T 00000000000000000000000000000000 P 229aee15bd3a41cabddf2b06290fe091 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:03:50.670302 20731 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 229aee15bd3a41cabddf2b06290fe091 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:50.671068 20731 raft_consensus.cc:686] T 00000000000000000000000000000000 P 229aee15bd3a41cabddf2b06290fe091 [term 1 LEADER]: Becoming Leader. State: Replica: 229aee15bd3a41cabddf2b06290fe091, State: Running, Role: LEADER
I20260620 23:03:50.671577 20731 consensus_queue.cc:237] T 00000000000000000000000000000000 P 229aee15bd3a41cabddf2b06290fe091 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "229aee15bd3a41cabddf2b06290fe091" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37103 } }
I20260620 23:03:50.672261 20726 sys_catalog.cc:564] T 00000000000000000000000000000000 P 229aee15bd3a41cabddf2b06290fe091 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:03:50.674336 20732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 229aee15bd3a41cabddf2b06290fe091 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "229aee15bd3a41cabddf2b06290fe091" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "229aee15bd3a41cabddf2b06290fe091" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37103 } } }
I20260620 23:03:50.674782 20732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 229aee15bd3a41cabddf2b06290fe091 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:50.674365 20733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 229aee15bd3a41cabddf2b06290fe091 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 229aee15bd3a41cabddf2b06290fe091. Latest consensus state: current_term: 1 leader_uuid: "229aee15bd3a41cabddf2b06290fe091" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "229aee15bd3a41cabddf2b06290fe091" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37103 } } }
I20260620 23:03:50.674952 20733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 229aee15bd3a41cabddf2b06290fe091 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:50.675658 20736 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:03:50.680775 20736 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:03:50.689184 20736 catalog_manager.cc:1347] Generated new cluster ID: 509108ac578549f7967f627976fde361
I20260620 23:03:50.689301 20736 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:03:50.704073 20736 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:03:50.705920 20736 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:03:50.715658 20736 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 229aee15bd3a41cabddf2b06290fe091: Generated new TSK 0
I20260620 23:03:50.716794 20736 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:03:50.728823 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:0
--local_ip_for_outbound_sockets=127.2.237.1
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:37103
--builtin_ntp_servers=127.2.237.20:39295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 23:03:51.057657 20750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:51.057997 20750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:51.058177 20750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:51.065704 20750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:51.065850 20750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:51.066030 20750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 23:03:51.075387 20750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:39295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:37103
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:03:51.076901 20750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:51.078812 20750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:51.092468 20755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:51.092551 20756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:51.092521 20758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:51.093375 20750 server_base.cc:1034] running on GCE node
I20260620 23:03:51.093972 20750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:51.095049 20750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:51.096304 20750 hybrid_clock.cc:648] HybridClock initialized: now 1781996631096245 us; error 60 us; skew 500 ppm
I20260620 23:03:51.096666 20750 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 23:03:51.099263 20750 webserver.cc:460] Webserver started at http://127.2.237.1:42697/ using document root <none> and password file <none>
I20260620 23:03:51.100016 20750 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:51.100172 20750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:51.100525 20750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:51.103667 20750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-0/data/instance:
uuid: "b7bd2cd2ecfb4df2b94880e60ff0f352"
format_stamp: "Formatted at 2026-06-20 23:03:51 on dist-test-slave-f63b"
server_key: "a6c88a9610389a726c46adb879fa8a89"
server_key_iv: "31aecd367d0766b2276172ab54defcf6"
server_key_version: "encryptionkey@0"
I20260620 23:03:51.104544 20750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance:
uuid: "b7bd2cd2ecfb4df2b94880e60ff0f352"
format_stamp: "Formatted at 2026-06-20 23:03:51 on dist-test-slave-f63b"
server_key: "a6c88a9610389a726c46adb879fa8a89"
server_key_iv: "31aecd367d0766b2276172ab54defcf6"
server_key_version: "encryptionkey@0"
I20260620 23:03:51.110021 20750 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260620 23:03:51.113794 20764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:51.115239 20750 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260620 23:03:51.115422 20750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "b7bd2cd2ecfb4df2b94880e60ff0f352"
format_stamp: "Formatted at 2026-06-20 23:03:51 on dist-test-slave-f63b"
server_key: "a6c88a9610389a726c46adb879fa8a89"
server_key_iv: "31aecd367d0766b2276172ab54defcf6"
server_key_version: "encryptionkey@0"
I20260620 23:03:51.115670 20750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:51.146422 20750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:51.147409 20750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:51.147708 20750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:51.148736 20750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:51.150213 20750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:51.150341 20750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:51.150475 20750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:51.150568 20750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:51.184253 20750 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:40523
I20260620 23:03:51.184345 20876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:40523 every 8 connection(s)
I20260620 23:03:51.185536 20750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 23:03:51.193109 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 20750
I20260620 23:03:51.193275 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance
I20260620 23:03:51.193727 2996 external_mini_cluster.cc:1428] Setting key 8ce2a0bc3a12b058466c879253d0a0a3
I20260620 23:03:51.196373 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:0
--local_ip_for_outbound_sockets=127.2.237.2
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:37103
--builtin_ntp_servers=127.2.237.20:39295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 23:03:51.198633 20879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:51.188170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:53161 (local address 127.2.237.62:37103)
0620 23:03:51.188625 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:51.188630 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:51.189689 (+ 1059us) server_negotiation.cc:407] Connection header received
0620 23:03:51.191583 (+ 1894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:51.191589 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:51.191681 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:51.191805 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:51.192948 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:51.193758 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:51.195714 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:51.196135 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:51.196271 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:51.197723 (+ 1452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:51.197728 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:51.197731 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:51.197748 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:51.197789 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:51.197801 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:51.197899 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:51.198121 (+ 222us) server_negotiation.cc:299] Negotiation successful
0620 23:03:51.198221 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":247,"threads_started":1}
I20260620 23:03:51.199756 20880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:51.188607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37103 (local address 127.2.237.1:53161)
0620 23:03:51.189517 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:51.189553 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:51.191298 (+ 1745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:51.192046 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:51.192055 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:51.192142 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:51.192737 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:51.192774 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:51.193931 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:51.193938 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:51.195577 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:51.195589 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:51.196356 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:51.196363 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:51.196662 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:51.197393 (+ 731us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:03:51.197395 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:51.197452 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:03:51.197530 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:51.197898 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:51.197911 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:51.197922 (+ 11us) client_negotiation.cc:769] Sending connection context
0620 23:03:51.198110 (+ 188us) client_negotiation.cc:240] Negotiation successful
0620 23:03:51.198532 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":296,"threads_started":1}
I20260620 23:03:51.201236 20877 heartbeater.cc:347] Connected to a master server at 127.2.237.62:37103
I20260620 23:03:51.201566 20877 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:51.202457 20877 heartbeater.cc:511] Master 127.2.237.62:37103 requested a full tablet report, sending...
I20260620 23:03:51.204964 20691 ts_manager.cc:194] Registered new tserver with Master: b7bd2cd2ecfb4df2b94880e60ff0f352 (127.2.237.1:40523)
I20260620 23:03:51.206710 20691 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.237.1:53161
W20260620 23:03:51.538599 20881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:51.538923 20881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:51.539094 20881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:51.546271 20881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:51.546403 20881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:51.546598 20881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 23:03:51.555605 20881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:39295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:37103
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:03:51.556910 20881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:51.558701 20881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:51.571835 20886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:51.571966 20889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:51.572093 20887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:51.573226 20881 server_base.cc:1034] running on GCE node
I20260620 23:03:51.573807 20881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:51.574855 20881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:51.576087 20881 hybrid_clock.cc:648] HybridClock initialized: now 1781996631576036 us; error 41 us; skew 500 ppm
I20260620 23:03:51.576397 20881 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 23:03:51.578730 20881 webserver.cc:460] Webserver started at http://127.2.237.2:34261/ using document root <none> and password file <none>
I20260620 23:03:51.579406 20881 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:51.579535 20881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:51.579849 20881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:51.582852 20881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-1/data/instance:
uuid: "df63b7467dc64b1d97c695e477294fea"
format_stamp: "Formatted at 2026-06-20 23:03:51 on dist-test-slave-f63b"
server_key: "5b908a14d459acad5c9bd04553be6c3f"
server_key_iv: "963368987c593a63dac6754a2cd5d59a"
server_key_version: "encryptionkey@0"
I20260620 23:03:51.583645 20881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance:
uuid: "df63b7467dc64b1d97c695e477294fea"
format_stamp: "Formatted at 2026-06-20 23:03:51 on dist-test-slave-f63b"
server_key: "5b908a14d459acad5c9bd04553be6c3f"
server_key_iv: "963368987c593a63dac6754a2cd5d59a"
server_key_version: "encryptionkey@0"
I20260620 23:03:51.588685 20881 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260620 23:03:51.592274 20895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:51.593678 20881 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 23:03:51.593848 20881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "df63b7467dc64b1d97c695e477294fea"
format_stamp: "Formatted at 2026-06-20 23:03:51 on dist-test-slave-f63b"
server_key: "5b908a14d459acad5c9bd04553be6c3f"
server_key_iv: "963368987c593a63dac6754a2cd5d59a"
server_key_version: "encryptionkey@0"
I20260620 23:03:51.594101 20881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:51.606468 20881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:51.607393 20881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:51.607694 20881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:51.608703 20881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:51.610144 20881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:51.610265 20881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:51.610404 20881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:51.610494 20881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:51.643822 20881 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:42981
I20260620 23:03:51.643839 21007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:42981 every 8 connection(s)
I20260620 23:03:51.645419 20881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 23:03:51.652519 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 20881
I20260620 23:03:51.652727 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance
I20260620 23:03:51.653251 2996 external_mini_cluster.cc:1428] Setting key 71baa03efe73868776b1fa6f79944615
I20260620 23:03:51.656423 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:0
--local_ip_for_outbound_sockets=127.2.237.3
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:37103
--builtin_ntp_servers=127.2.237.20:39295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 23:03:51.659471 20879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:51.648190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:51433 (local address 127.2.237.62:37103)
0620 23:03:51.648388 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:51.648393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:51.649887 (+ 1494us) server_negotiation.cc:407] Connection header received
0620 23:03:51.651722 (+ 1835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:51.651728 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:51.651808 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:51.651924 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:51.653284 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:51.654270 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:51.656360 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:51.656897 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:51.657058 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:51.658613 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:51.658620 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:51.658625 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:51.658652 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:51.658685 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:51.658698 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:51.658799 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:51.659067 (+ 268us) server_negotiation.cc:299] Negotiation successful
0620 23:03:51.659168 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260620 23:03:51.660668 21010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:51.648634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37103 (local address 127.2.237.2:51433)
0620 23:03:51.649707 (+ 1073us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:51.649751 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:51.651465 (+ 1714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:51.652203 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:51.652212 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:51.652315 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:51.653079 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:51.653101 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:51.654452 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:51.654458 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:51.656196 (+ 1738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:51.656209 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:51.657151 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:51.657159 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:51.657426 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:51.658231 (+ 805us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:03:51.658234 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:51.658282 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:03:51.658362 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:51.658799 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:51.658813 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:51.658824 (+ 11us) client_negotiation.cc:769] Sending connection context
0620 23:03:51.659067 (+ 243us) client_negotiation.cc:240] Negotiation successful
0620 23:03:51.659476 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":358,"threads_started":1}
I20260620 23:03:51.662235 21008 heartbeater.cc:347] Connected to a master server at 127.2.237.62:37103
I20260620 23:03:51.662549 21008 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:51.663364 21008 heartbeater.cc:511] Master 127.2.237.62:37103 requested a full tablet report, sending...
I20260620 23:03:51.664961 20691 ts_manager.cc:194] Registered new tserver with Master: df63b7467dc64b1d97c695e477294fea (127.2.237.2:42981)
I20260620 23:03:51.665800 20691 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.237.2:51433
W20260620 23:03:51.989989 21011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:51.990352 21011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:51.990518 21011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:51.998394 21011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:51.998626 21011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:51.998816 21011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 23:03:52.008379 21011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:39295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:37103
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:03:52.009800 21011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:52.011734 21011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:52.025574 21017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:52.025655 21019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:52.025684 21016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:52.026728 21011 server_base.cc:1034] running on GCE node
I20260620 23:03:52.027318 21011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:52.028281 21011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:52.029486 21011 hybrid_clock.cc:648] HybridClock initialized: now 1781996632029439 us; error 30 us; skew 500 ppm
I20260620 23:03:52.029771 21011 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 23:03:52.032250 21011 webserver.cc:460] Webserver started at http://127.2.237.3:40605/ using document root <none> and password file <none>
I20260620 23:03:52.032953 21011 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:52.033039 21011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:52.033296 21011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:52.036371 21011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-2/data/instance:
uuid: "d1ab6c9eca4c4428a4b015ad7814c3ac"
format_stamp: "Formatted at 2026-06-20 23:03:52 on dist-test-slave-f63b"
server_key: "04d843687aad1e187104e3f66534ee03"
server_key_iv: "bcf51896c137e56bfe00b4d6163225d9"
server_key_version: "encryptionkey@0"
I20260620 23:03:52.037174 21011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance:
uuid: "d1ab6c9eca4c4428a4b015ad7814c3ac"
format_stamp: "Formatted at 2026-06-20 23:03:52 on dist-test-slave-f63b"
server_key: "04d843687aad1e187104e3f66534ee03"
server_key_iv: "bcf51896c137e56bfe00b4d6163225d9"
server_key_version: "encryptionkey@0"
I20260620 23:03:52.042461 21011 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260620 23:03:52.046216 21025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:52.047948 21011 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 23:03:52.048154 21011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "d1ab6c9eca4c4428a4b015ad7814c3ac"
format_stamp: "Formatted at 2026-06-20 23:03:52 on dist-test-slave-f63b"
server_key: "04d843687aad1e187104e3f66534ee03"
server_key_iv: "bcf51896c137e56bfe00b4d6163225d9"
server_key_version: "encryptionkey@0"
I20260620 23:03:52.048399 21011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:52.073724 21011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:52.074676 21011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:52.074958 21011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:52.076009 21011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:52.077766 21011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:52.077889 21011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:52.077989 21011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:52.078060 21011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:52.112572 21011 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:44501
I20260620 23:03:52.112697 21137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:44501 every 8 connection(s)
I20260620 23:03:52.113916 21011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 23:03:52.123121 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 21011
I20260620 23:03:52.123299 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance
I20260620 23:03:52.123744 2996 external_mini_cluster.cc:1428] Setting key 2ef26942508734325b2ec9dc4f1ec429
I20260620 23:03:52.127350 20879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:52.116826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:34603 (local address 127.2.237.62:37103)
0620 23:03:52.117028 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:52.117032 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:52.118513 (+ 1481us) server_negotiation.cc:407] Connection header received
0620 23:03:52.120566 (+ 2053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:52.120570 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:52.120643 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:52.120735 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:52.121811 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.122667 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.124432 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.124900 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.125051 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:52.126475 (+ 1424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:52.126488 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:52.126492 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:52.126509 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:52.126546 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:52.126559 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:52.126690 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:52.126921 (+ 231us) server_negotiation.cc:299] Negotiation successful
0620 23:03:52.127024 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260620 23:03:52.128450 21140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:52.117333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37103 (local address 127.2.237.3:34603)
0620 23:03:52.118336 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:52.118378 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:52.120246 (+ 1868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:52.120951 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:52.120959 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:52.121068 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:52.121621 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:52.121636 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.122813 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.122817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:52.124281 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:52.124293 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.125155 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.125169 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:52.125398 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:52.126131 (+ 733us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:03:52.126133 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:52.126176 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:03:52.126263 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:52.126688 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:52.126701 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:52.126712 (+ 11us) client_negotiation.cc:769] Sending connection context
0620 23:03:52.126907 (+ 195us) client_negotiation.cc:240] Negotiation successful
0620 23:03:52.127327 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":326,"threads_started":1}
I20260620 23:03:52.129828 21138 heartbeater.cc:347] Connected to a master server at 127.2.237.62:37103
I20260620 23:03:52.130143 21138 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:52.130900 21138 heartbeater.cc:511] Master 127.2.237.62:37103 requested a full tablet report, sending...
I20260620 23:03:52.132292 20691 ts_manager.cc:194] Registered new tserver with Master: d1ab6c9eca4c4428a4b015ad7814c3ac (127.2.237.3:44501)
I20260620 23:03:52.133072 20691 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.237.3:34603
I20260620 23:03:52.138831 2996 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:03:52.148859 20879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:52.142977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57538 (local address 127.2.237.62:37103)
0620 23:03:52.143267 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:52.143273 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:52.143570 (+ 297us) server_negotiation.cc:407] Connection header received
0620 23:03:52.143710 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:52.143714 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:52.143791 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:52.143877 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:52.144417 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.145119 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.146498 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.146959 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.147109 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:52.148151 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:52.148156 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:52.148159 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:52.148177 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:52.148208 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:52.148220 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:52.148295 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:52.148445 (+ 150us) server_negotiation.cc:299] Negotiation successful
0620 23:03:52.148561 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260620 23:03:52.152704 20691 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57538:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:03:52.159778 20691 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:03:52.194984 21152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:52.172558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:42981 (local address 127.0.0.1:37144)
0620 23:03:52.179710 (+ 7152us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:52.179757 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:52.179939 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:52.186992 (+ 7053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:52.186999 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:52.187031 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:52.187474 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:52.187491 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.189680 (+ 2189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.189691 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:52.192106 (+ 2415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:52.192120 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.193953 (+ 1833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.193959 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:52.194175 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:52.194256 (+ 81us) client_negotiation.cc:769] Sending connection context
0620 23:03:52.194368 (+ 112us) client_negotiation.cc:240] Negotiation successful
0620 23:03:52.194625 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6856,"thread_start_us":6690,"threads_started":1}
I20260620 23:03:52.195698 21150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:52.172603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37144 (local address 127.2.237.2:42981)
0620 23:03:52.185134 (+ 12531us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:52.185148 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:52.185192 (+ 44us) server_negotiation.cc:407] Connection header received
0620 23:03:52.185342 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:52.185354 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:52.185696 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:52.187731 (+ 2035us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:52.187990 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.189526 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.192340 (+ 2814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.193757 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.194729 (+ 972us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:52.194989 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:52.195070 (+ 81us) server_negotiation.cc:299] Negotiation successful
0620 23:03:52.195295 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12288,"thread_start_us":13997,"threads_started":1}
I20260620 23:03:52.199453 21155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:52.185670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:40523 (local address 127.0.0.1:49308)
0620 23:03:52.187885 (+ 2215us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:52.187903 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:52.188009 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:52.190529 (+ 2520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:52.190534 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:52.190560 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:52.191016 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:52.191029 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.194282 (+ 3253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.194286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:52.196370 (+ 2084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:52.196384 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.198525 (+ 2141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.198531 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:52.198871 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:52.198906 (+ 35us) client_negotiation.cc:769] Sending connection context
0620 23:03:52.199007 (+ 101us) client_negotiation.cc:240] Negotiation successful
0620 23:03:52.199135 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2102,"thread_start_us":1958,"threads_started":1}
I20260620 23:03:52.200248 21151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:52.172634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:44501 (local address 127.0.0.1:39754)
0620 23:03:52.186372 (+ 13738us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:52.186390 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:52.186511 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:52.192281 (+ 5770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:52.192287 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:52.192316 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:52.192830 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:52.192843 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.195186 (+ 2343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.195192 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:52.197430 (+ 2238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:52.197446 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.199501 (+ 2055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.199508 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:52.199699 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:52.199740 (+ 41us) client_negotiation.cc:769] Sending connection context
0620 23:03:52.199813 (+ 73us) client_negotiation.cc:240] Negotiation successful
0620 23:03:52.199931 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13608,"thread_start_us":13455,"threads_started":1}
I20260620 23:03:52.200402 21154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:52.180561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49308 (local address 127.2.237.1:40523)
0620 23:03:52.188539 (+ 7978us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:52.188550 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:52.188582 (+ 32us) server_negotiation.cc:407] Connection header received
0620 23:03:52.188687 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:52.188695 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:52.188929 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:52.189129 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:52.191316 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.194133 (+ 2817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.196595 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.198289 (+ 1694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.198485 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:52.198925 (+ 440us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:52.199864 (+ 939us) server_negotiation.cc:299] Negotiation successful
0620 23:03:52.200063 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7789,"thread_start_us":7639,"threads_started":1}
I20260620 23:03:52.199972 20943 tablet_service.cc:1467] Processing CreateTablet for tablet d14e20e0d4fa4125881cc724157bbd21 (DEFAULT_TABLE table=test-table [id=1416504419d34da89ed53d70d979e221]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:52.201252 21153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:52.173276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39754 (local address 127.2.237.3:44501)
0620 23:03:52.191437 (+ 18161us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:52.191449 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:52.191489 (+ 40us) server_negotiation.cc:407] Connection header received
0620 23:03:52.191596 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:52.191604 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:52.191862 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:52.193125 (+ 1263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:52.193433 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.195005 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.197676 (+ 2671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.199253 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.200229 (+ 976us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:52.200585 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:52.200663 (+ 78us) server_negotiation.cc:299] Negotiation successful
0620 23:03:52.200834 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17960,"thread_start_us":18732,"threads_started":1}
I20260620 23:03:52.201541 20943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d14e20e0d4fa4125881cc724157bbd21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:52.204994 21073 tablet_service.cc:1467] Processing CreateTablet for tablet d14e20e0d4fa4125881cc724157bbd21 (DEFAULT_TABLE table=test-table [id=1416504419d34da89ed53d70d979e221]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:52.204959 20812 tablet_service.cc:1467] Processing CreateTablet for tablet d14e20e0d4fa4125881cc724157bbd21 (DEFAULT_TABLE table=test-table [id=1416504419d34da89ed53d70d979e221]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:52.206382 21073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d14e20e0d4fa4125881cc724157bbd21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:52.206432 20812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d14e20e0d4fa4125881cc724157bbd21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:52.209717 20877 heartbeater.cc:503] Master 127.2.237.62:37103 was elected leader, sending a full tablet report...
I20260620 23:03:52.218139 21156 tablet_bootstrap.cc:492] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea: Bootstrap starting.
I20260620 23:03:52.220153 21157 tablet_bootstrap.cc:492] T d14e20e0d4fa4125881cc724157bbd21 P b7bd2cd2ecfb4df2b94880e60ff0f352: Bootstrap starting.
I20260620 23:03:52.223006 21156 tablet_bootstrap.cc:654] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:52.223304 21158 tablet_bootstrap.cc:492] T d14e20e0d4fa4125881cc724157bbd21 P d1ab6c9eca4c4428a4b015ad7814c3ac: Bootstrap starting.
I20260620 23:03:52.224385 21157 tablet_bootstrap.cc:654] T d14e20e0d4fa4125881cc724157bbd21 P b7bd2cd2ecfb4df2b94880e60ff0f352: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:52.224701 21156 log.cc:824] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:52.225636 21157 log.cc:824] T d14e20e0d4fa4125881cc724157bbd21 P b7bd2cd2ecfb4df2b94880e60ff0f352: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:52.226722 21158 tablet_bootstrap.cc:654] T d14e20e0d4fa4125881cc724157bbd21 P d1ab6c9eca4c4428a4b015ad7814c3ac: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:52.227895 21158 log.cc:824] T d14e20e0d4fa4125881cc724157bbd21 P d1ab6c9eca4c4428a4b015ad7814c3ac: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:52.228672 21156 tablet_bootstrap.cc:492] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea: No bootstrap required, opened a new log
I20260620 23:03:52.229000 21156 ts_tablet_manager.cc:1397] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260620 23:03:52.235247 21158 tablet_bootstrap.cc:492] T d14e20e0d4fa4125881cc724157bbd21 P d1ab6c9eca4c4428a4b015ad7814c3ac: No bootstrap required, opened a new log
I20260620 23:03:52.235394 21157 tablet_bootstrap.cc:492] T d14e20e0d4fa4125881cc724157bbd21 P b7bd2cd2ecfb4df2b94880e60ff0f352: No bootstrap required, opened a new log
I20260620 23:03:52.235608 21158 ts_tablet_manager.cc:1397] T d14e20e0d4fa4125881cc724157bbd21 P d1ab6c9eca4c4428a4b015ad7814c3ac: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.010s
I20260620 23:03:52.235671 21157 ts_tablet_manager.cc:1397] T d14e20e0d4fa4125881cc724157bbd21 P b7bd2cd2ecfb4df2b94880e60ff0f352: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.010s
I20260620 23:03:52.240942 21156 raft_consensus.cc:348] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df63b7467dc64b1d97c695e477294fea" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 42981 } } peers { permanent_uuid: "d1ab6c9eca4c4428a4b015ad7814c3ac" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44501 } } peers { permanent_uuid: "b7bd2cd2ecfb4df2b94880e60ff0f352" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 40523 } }
I20260620 23:03:52.241437 21156 raft_consensus.cc:374] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:52.241580 21156 raft_consensus.cc:729] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df63b7467dc64b1d97c695e477294fea, State: Initialized, Role: FOLLOWER
I20260620 23:03:52.242236 21156 consensus_queue.cc:260] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df63b7467dc64b1d97c695e477294fea" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 42981 } } peers { permanent_uuid: "d1ab6c9eca4c4428a4b015ad7814c3ac" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44501 } } peers { permanent_uuid: "b7bd2cd2ecfb4df2b94880e60ff0f352" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 40523 } }
I20260620 23:03:52.243542 21008 heartbeater.cc:503] Master 127.2.237.62:37103 was elected leader, sending a full tablet report...
I20260620 23:03:52.244179 21156 ts_tablet_manager.cc:1428] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260620 23:03:52.245339 21158 raft_consensus.cc:348] T d14e20e0d4fa4125881cc724157bbd21 P d1ab6c9eca4c4428a4b015ad7814c3ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df63b7467dc64b1d97c695e477294fea" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 42981 } } peers { permanent_uuid: "d1ab6c9eca4c4428a4b015ad7814c3ac" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44501 } } peers { permanent_uuid: "b7bd2cd2ecfb4df2b94880e60ff0f352" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 40523 } }
I20260620 23:03:52.245834 21158 raft_consensus.cc:374] T d14e20e0d4fa4125881cc724157bbd21 P d1ab6c9eca4c4428a4b015ad7814c3ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:52.245978 21158 raft_consensus.cc:729] T d14e20e0d4fa4125881cc724157bbd21 P d1ab6c9eca4c4428a4b015ad7814c3ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1ab6c9eca4c4428a4b015ad7814c3ac, State: Initialized, Role: FOLLOWER
I20260620 23:03:52.245824 21157 raft_consensus.cc:348] T d14e20e0d4fa4125881cc724157bbd21 P b7bd2cd2ecfb4df2b94880e60ff0f352 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df63b7467dc64b1d97c695e477294fea" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 42981 } } peers { permanent_uuid: "d1ab6c9eca4c4428a4b015ad7814c3ac" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44501 } } peers { permanent_uuid: "b7bd2cd2ecfb4df2b94880e60ff0f352" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 40523 } }
I20260620 23:03:52.246289 21157 raft_consensus.cc:374] T d14e20e0d4fa4125881cc724157bbd21 P b7bd2cd2ecfb4df2b94880e60ff0f352 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:52.246449 21157 raft_consensus.cc:729] T d14e20e0d4fa4125881cc724157bbd21 P b7bd2cd2ecfb4df2b94880e60ff0f352 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7bd2cd2ecfb4df2b94880e60ff0f352, State: Initialized, Role: FOLLOWER
I20260620 23:03:52.246727 21158 consensus_queue.cc:260] T d14e20e0d4fa4125881cc724157bbd21 P d1ab6c9eca4c4428a4b015ad7814c3ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df63b7467dc64b1d97c695e477294fea" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 42981 } } peers { permanent_uuid: "d1ab6c9eca4c4428a4b015ad7814c3ac" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44501 } } peers { permanent_uuid: "b7bd2cd2ecfb4df2b94880e60ff0f352" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 40523 } }
I20260620 23:03:52.247205 21157 consensus_queue.cc:260] T d14e20e0d4fa4125881cc724157bbd21 P b7bd2cd2ecfb4df2b94880e60ff0f352 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df63b7467dc64b1d97c695e477294fea" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 42981 } } peers { permanent_uuid: "d1ab6c9eca4c4428a4b015ad7814c3ac" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44501 } } peers { permanent_uuid: "b7bd2cd2ecfb4df2b94880e60ff0f352" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 40523 } }
I20260620 23:03:52.247951 21138 heartbeater.cc:503] Master 127.2.237.62:37103 was elected leader, sending a full tablet report...
I20260620 23:03:52.248147 21162 raft_consensus.cc:482] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:03:52.248454 21162 raft_consensus.cc:504] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df63b7467dc64b1d97c695e477294fea" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 42981 } } peers { permanent_uuid: "d1ab6c9eca4c4428a4b015ad7814c3ac" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44501 } } peers { permanent_uuid: "b7bd2cd2ecfb4df2b94880e60ff0f352" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 40523 } }
I20260620 23:03:52.248709 21158 ts_tablet_manager.cc:1428] T d14e20e0d4fa4125881cc724157bbd21 P d1ab6c9eca4c4428a4b015ad7814c3ac: Time spent starting tablet: real 0.013s user 0.010s sys 0.003s
I20260620 23:03:52.255084 21162 leader_election.cc:288] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1ab6c9eca4c4428a4b015ad7814c3ac (127.2.237.3:44501), b7bd2cd2ecfb4df2b94880e60ff0f352 (127.2.237.1:40523)
I20260620 23:03:52.257637 21157 ts_tablet_manager.cc:1428] T d14e20e0d4fa4125881cc724157bbd21 P b7bd2cd2ecfb4df2b94880e60ff0f352: Time spent starting tablet: real 0.022s user 0.014s sys 0.004s
I20260620 23:03:52.262971 21153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:52.251078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:51889 (local address 127.2.237.3:44501)
0620 23:03:52.251268 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:52.251274 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:52.256144 (+ 4870us) server_negotiation.cc:407] Connection header received
0620 23:03:52.256420 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:52.256425 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:52.256497 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:52.256558 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:52.257358 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.258407 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.261111 (+ 2704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.262023 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.262183 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:52.262287 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:52.262611 (+ 324us) server_negotiation.cc:299] Negotiation successful
0620 23:03:52.262740 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 23:03:52.263043 21165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:52.250685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:44501 (local address 127.2.237.2:51889)
0620 23:03:52.255984 (+ 5299us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:52.256007 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:52.256256 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:52.256765 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:52.256769 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:52.256794 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:52.257191 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:52.257204 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.258678 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.258686 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:52.260909 (+ 2223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:52.260926 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.262234 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.262241 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:52.262421 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:52.262445 (+ 24us) client_negotiation.cc:769] Sending connection context
0620 23:03:52.262565 (+ 120us) client_negotiation.cc:240] Negotiation successful
0620 23:03:52.262711 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5117,"thread_start_us":4925,"threads_started":1}
I20260620 23:03:52.264017 21093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d14e20e0d4fa4125881cc724157bbd21" candidate_uuid: "df63b7467dc64b1d97c695e477294fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1ab6c9eca4c4428a4b015ad7814c3ac" is_pre_election: true
I20260620 23:03:52.264678 21093 raft_consensus.cc:2454] T d14e20e0d4fa4125881cc724157bbd21 P d1ab6c9eca4c4428a4b015ad7814c3ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df63b7467dc64b1d97c695e477294fea in term 0.
I20260620 23:03:52.265146 21166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:52.255600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:40523 (local address 127.2.237.2:47017)
0620 23:03:52.258515 (+ 2915us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:52.258531 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:52.258713 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:52.259237 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:52.259242 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:52.259287 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:52.259648 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:52.259657 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.260835 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.260841 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:52.262667 (+ 1826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:52.262682 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.264489 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.264495 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:52.264634 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:52.264657 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 23:03:52.264731 (+ 74us) client_negotiation.cc:240] Negotiation successful
0620 23:03:52.264816 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2791,"thread_start_us":2583,"threads_started":1}
I20260620 23:03:52.265192 21154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:52.256218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:47017 (local address 127.2.237.1:40523)
0620 23:03:52.256434 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:52.256440 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:52.258656 (+ 2216us) server_negotiation.cc:407] Connection header received
0620 23:03:52.258871 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:52.258879 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:52.258994 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:52.259089 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:52.259814 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.260649 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.262867 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.264244 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.264398 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:52.264516 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:52.264793 (+ 277us) server_negotiation.cc:299] Negotiation successful
0620 23:03:52.264938 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 23:03:52.265476 20896 leader_election.cc:302] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1ab6c9eca4c4428a4b015ad7814c3ac, df63b7467dc64b1d97c695e477294fea; no voters:
I20260620 23:03:52.265980 21162 raft_consensus.cc:2781] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:03:52.266191 21162 raft_consensus.cc:482] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:03:52.266310 21162 raft_consensus.cc:3037] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:52.266564 20832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d14e20e0d4fa4125881cc724157bbd21" candidate_uuid: "df63b7467dc64b1d97c695e477294fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7bd2cd2ecfb4df2b94880e60ff0f352" is_pre_election: true
I20260620 23:03:52.267172 20832 raft_consensus.cc:2454] T d14e20e0d4fa4125881cc724157bbd21 P b7bd2cd2ecfb4df2b94880e60ff0f352 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df63b7467dc64b1d97c695e477294fea in term 0.
I20260620 23:03:52.269093 21162 raft_consensus.cc:504] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df63b7467dc64b1d97c695e477294fea" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 42981 } } peers { permanent_uuid: "d1ab6c9eca4c4428a4b015ad7814c3ac" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44501 } } peers { permanent_uuid: "b7bd2cd2ecfb4df2b94880e60ff0f352" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 40523 } }
I20260620 23:03:52.269881 21162 leader_election.cc:288] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea [CANDIDATE]: Term 1 election: Requested vote from peers d1ab6c9eca4c4428a4b015ad7814c3ac (127.2.237.3:44501), b7bd2cd2ecfb4df2b94880e60ff0f352 (127.2.237.1:40523)
I20260620 23:03:52.270339 21093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d14e20e0d4fa4125881cc724157bbd21" candidate_uuid: "df63b7467dc64b1d97c695e477294fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1ab6c9eca4c4428a4b015ad7814c3ac"
I20260620 23:03:52.270363 20832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d14e20e0d4fa4125881cc724157bbd21" candidate_uuid: "df63b7467dc64b1d97c695e477294fea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7bd2cd2ecfb4df2b94880e60ff0f352"
I20260620 23:03:52.270606 21093 raft_consensus.cc:3037] T d14e20e0d4fa4125881cc724157bbd21 P d1ab6c9eca4c4428a4b015ad7814c3ac [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:52.270633 20832 raft_consensus.cc:3037] T d14e20e0d4fa4125881cc724157bbd21 P b7bd2cd2ecfb4df2b94880e60ff0f352 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:52.273286 21093 raft_consensus.cc:2454] T d14e20e0d4fa4125881cc724157bbd21 P d1ab6c9eca4c4428a4b015ad7814c3ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df63b7467dc64b1d97c695e477294fea in term 1.
I20260620 23:03:52.273307 20832 raft_consensus.cc:2454] T d14e20e0d4fa4125881cc724157bbd21 P b7bd2cd2ecfb4df2b94880e60ff0f352 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df63b7467dc64b1d97c695e477294fea in term 1.
I20260620 23:03:52.273756 20899 leader_election.cc:302] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7bd2cd2ecfb4df2b94880e60ff0f352, df63b7467dc64b1d97c695e477294fea; no voters:
I20260620 23:03:52.274111 21162 raft_consensus.cc:2781] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:52.274649 21162 raft_consensus.cc:686] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea [term 1 LEADER]: Becoming Leader. State: Replica: df63b7467dc64b1d97c695e477294fea, State: Running, Role: LEADER
I20260620 23:03:52.275170 21162 consensus_queue.cc:237] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df63b7467dc64b1d97c695e477294fea" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 42981 } } peers { permanent_uuid: "d1ab6c9eca4c4428a4b015ad7814c3ac" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44501 } } peers { permanent_uuid: "b7bd2cd2ecfb4df2b94880e60ff0f352" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 40523 } }
I20260620 23:03:52.280328 20690 catalog_manager.cc:5495] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea reported cstate change: term changed from 0 to 1, leader changed from <none> to df63b7467dc64b1d97c695e477294fea (127.2.237.2). New cstate: current_term: 1 leader_uuid: "df63b7467dc64b1d97c695e477294fea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df63b7467dc64b1d97c695e477294fea" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 42981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1ab6c9eca4c4428a4b015ad7814c3ac" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7bd2cd2ecfb4df2b94880e60ff0f352" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 40523 } health_report { overall_health: UNKNOWN } } }
I20260620 23:03:52.306057 21150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:52.300393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37146 (local address 127.2.237.2:42981)
0620 23:03:52.300624 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:52.300632 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:52.300653 (+ 21us) server_negotiation.cc:407] Connection header received
0620 23:03:52.300909 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:52.300913 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:52.301022 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:52.301180 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:03:52.301565 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.302273 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.303850 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.304276 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.304418 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:52.304946 (+ 528us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:03:52.305466 (+ 520us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:03:52.305643 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:52.305702 (+ 59us) server_negotiation.cc:299] Negotiation successful
0620 23:03:52.305788 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260620 23:03:52.314746 20832 raft_consensus.cc:1261] T d14e20e0d4fa4125881cc724157bbd21 P b7bd2cd2ecfb4df2b94880e60ff0f352 [term 1 FOLLOWER]: Refusing update from remote peer df63b7467dc64b1d97c695e477294fea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:52.314746 21093 raft_consensus.cc:1261] T d14e20e0d4fa4125881cc724157bbd21 P d1ab6c9eca4c4428a4b015ad7814c3ac [term 1 FOLLOWER]: Refusing update from remote peer df63b7467dc64b1d97c695e477294fea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:52.315850 21162 consensus_queue.cc:1035] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1ab6c9eca4c4428a4b015ad7814c3ac" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:03:52.316166 21167 consensus_queue.cc:1035] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7bd2cd2ecfb4df2b94880e60ff0f352" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 40523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:03:52.328680 21172 mvcc.cc:205] Tried to move back new op lower bound from 7299058205949661184 to 7299058205803253760. Current Snapshot: MvccSnapshot[applied={T|T < 7299058205949661184}]
I20260620 23:03:52.328840 21170 mvcc.cc:205] Tried to move back new op lower bound from 7299058205949661184 to 7299058205803253760. Current Snapshot: MvccSnapshot[applied={T|T < 7299058205949661184}]
I20260620 23:03:52.344918 20689 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:03:52.345220 20689 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:03:52.349385 20689 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 229aee15bd3a41cabddf2b06290fe091: Sending DeleteTablet for 3 replicas of tablet d14e20e0d4fa4125881cc724157bbd21
I20260620 23:03:52.351042 21073 tablet_service.cc:1514] Processing DeleteTablet for tablet d14e20e0d4fa4125881cc724157bbd21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:52 UTC) from {username='slave'} at 127.0.0.1:39754
I20260620 23:03:52.352439 20812 tablet_service.cc:1514] Processing DeleteTablet for tablet d14e20e0d4fa4125881cc724157bbd21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:52 UTC) from {username='slave'} at 127.0.0.1:49308
I20260620 23:03:52.352572 21179 tablet_replica.cc:330] T d14e20e0d4fa4125881cc724157bbd21 P d1ab6c9eca4c4428a4b015ad7814c3ac: stopping tablet replica
I20260620 23:03:52.352784 20943 tablet_service.cc:1514] Processing DeleteTablet for tablet d14e20e0d4fa4125881cc724157bbd21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:52 UTC) from {username='slave'} at 127.0.0.1:37144
I20260620 23:03:52.359270 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 20750
I20260620 23:03:52.368639 21181 tablet_replica.cc:330] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea: stopping tablet replica
I20260620 23:03:52.369302 21181 raft_consensus.cc:2229] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:03:52.369779 21179 raft_consensus.cc:2229] T d14e20e0d4fa4125881cc724157bbd21 P d1ab6c9eca4c4428a4b015ad7814c3ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:03:52.370337 21179 raft_consensus.cc:2258] T d14e20e0d4fa4125881cc724157bbd21 P d1ab6c9eca4c4428a4b015ad7814c3ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:52.370291 21181 raft_consensus.cc:2258] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:52.371106 21180 tablet_replica.cc:330] T d14e20e0d4fa4125881cc724157bbd21 P b7bd2cd2ecfb4df2b94880e60ff0f352: stopping tablet replica
I20260620 23:03:52.371649 21180 raft_consensus.cc:2229] T d14e20e0d4fa4125881cc724157bbd21 P b7bd2cd2ecfb4df2b94880e60ff0f352 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:03:52.372267 21180 raft_consensus.cc:2258] T d14e20e0d4fa4125881cc724157bbd21 P b7bd2cd2ecfb4df2b94880e60ff0f352 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:52.372926 21179 ts_tablet_manager.cc:1905] T d14e20e0d4fa4125881cc724157bbd21 P d1ab6c9eca4c4428a4b015ad7814c3ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:52.374856 21180 ts_tablet_manager.cc:1905] T d14e20e0d4fa4125881cc724157bbd21 P b7bd2cd2ecfb4df2b94880e60ff0f352: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:52.377794 21181 ts_tablet_manager.cc:1905] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:52.381863 21179 ts_tablet_manager.cc:1918] T d14e20e0d4fa4125881cc724157bbd21 P d1ab6c9eca4c4428a4b015ad7814c3ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:03:52.382050 21179 log.cc:1196] T d14e20e0d4fa4125881cc724157bbd21 P d1ab6c9eca4c4428a4b015ad7814c3ac: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-2/wal/wals/d14e20e0d4fa4125881cc724157bbd21
I20260620 23:03:52.382288 21154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:52.362634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49316 (local address 127.2.237.1:40523)
0620 23:03:52.362894 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:52.362903 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:52.373936 (+ 11033us) server_negotiation.cc:407] Connection header received
0620 23:03:52.374155 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:52.374161 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:52.374271 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:52.374394 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:52.375148 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.376318 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.378300 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.378935 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.379050 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:52.380507 (+ 1457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:52.380524 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:52.380543 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:52.381313 (+ 770us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:52.381377 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:52.381402 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:52.381489 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:52.381749 (+ 260us) server_negotiation.cc:299] Negotiation successful
0620 23:03:52.381880 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260620 23:03:52.382606 21179 ts_tablet_manager.cc:1939] T d14e20e0d4fa4125881cc724157bbd21 P d1ab6c9eca4c4428a4b015ad7814c3ac: Deleting consensus metadata
I20260620 23:03:52.383428 20872 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:03:52.384928 20675 catalog_manager.cc:4841] TS d1ab6c9eca4c4428a4b015ad7814c3ac (127.2.237.3:44501): tablet d14e20e0d4fa4125881cc724157bbd21 (table test-table [id=1416504419d34da89ed53d70d979e221]) successfully deleted
I20260620 23:03:52.389154 21181 ts_tablet_manager.cc:1918] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:03:52.389336 21181 log.cc:1196] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781996318807807-2996-0/minicluster-data/ts-1/wal/wals/d14e20e0d4fa4125881cc724157bbd21
I20260620 23:03:52.389897 21181 ts_tablet_manager.cc:1939] T d14e20e0d4fa4125881cc724157bbd21 P df63b7467dc64b1d97c695e477294fea: Deleting consensus metadata
I20260620 23:03:52.391861 20678 catalog_manager.cc:4841] TS df63b7467dc64b1d97c695e477294fea (127.2.237.2:42981): tablet d14e20e0d4fa4125881cc724157bbd21 (table test-table [id=1416504419d34da89ed53d70d979e221]) successfully deleted
I20260620 23:03:52.476140 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 20750
W20260620 23:03:52.500144 20678 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.237.1:40523 (error 108)
I20260620 23:03:52.500859 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 20881
I20260620 23:03:52.502436 21155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:52.501711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:40523 (local address 127.0.0.1:49320)
0620 23:03:52.502130 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:52.502257 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:40523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293}
W20260620 23:03:52.502764 20678 catalog_manager.cc:4584] TS b7bd2cd2ecfb4df2b94880e60ff0f352 (127.2.237.1:40523): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d14e20e0d4fa4125881cc724157bbd21: Network error: Client connection negotiation failed: client connection to 127.2.237.1:40523: connect: Connection refused (error 111)
I20260620 23:03:52.514160 21150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:52.501676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37156 (local address 127.2.237.2:42981)
0620 23:03:52.501955 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:52.501960 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:52.501979 (+ 19us) server_negotiation.cc:407] Connection header received
0620 23:03:52.502200 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:52.502205 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:52.502309 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:52.502434 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:52.503026 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.504221 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.505495 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.506177 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.506328 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:52.512283 (+ 5955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:52.512302 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:52.512322 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:52.513204 (+ 882us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:52.513272 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:52.513302 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:52.513430 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:52.513630 (+ 200us) server_negotiation.cc:299] Negotiation successful
0620 23:03:52.513758 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174}
I20260620 23:03:52.515173 21003 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:03:52.556418 21155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:52.555910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:40523 (local address 127.0.0.1:49326)
0620 23:03:52.556147 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:52.556253 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:40523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260620 23:03:52.557600 21155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:52.557204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:40523 (local address 127.0.0.1:49330)
0620 23:03:52.557389 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:52.557472 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:40523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260620 23:03:52.618508 21155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:52.617883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:40523 (local address 127.0.0.1:49332)
0620 23:03:52.618210 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:52.618311 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:40523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188}
I20260620 23:03:52.619804 21155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:52.619373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:40523 (local address 127.0.0.1:49340)
0620 23:03:52.619557 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:52.619645 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:40523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260620 23:03:52.629585 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 20881
I20260620 23:03:52.648347 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 21011
I20260620 23:03:52.655853 21153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:52.649107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39768 (local address 127.2.237.3:44501)
0620 23:03:52.649295 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:52.649301 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:52.649416 (+ 115us) server_negotiation.cc:407] Connection header received
0620 23:03:52.649649 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:52.649655 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:52.649748 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:52.649866 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:52.650298 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.651132 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.652609 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:52.653037 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:52.653176 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:52.654334 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:52.654343 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:52.654358 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:52.655105 (+ 747us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:52.655153 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:52.655172 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:52.655305 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:52.655440 (+ 135us) server_negotiation.cc:299] Negotiation successful
0620 23:03:52.655547 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260620 23:03:52.656483 21133 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:03:52.700562 21155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:52.700021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:40523 (local address 127.0.0.1:49346)
0620 23:03:52.700227 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:52.700349 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:40523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260620 23:03:52.702076 21155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:52.701506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:40523 (local address 127.0.0.1:49356)
0620 23:03:52.701826 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:52.701929 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:40523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
I20260620 23:03:52.748775 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 21011
I20260620 23:03:52.765573 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 20660
I20260620 23:03:52.766849 20721 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:03:52.877197 21155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:52.876563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:40523 (local address 127.0.0.1:49366)
0620 23:03:52.876825 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:52.876980 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:40523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260620 23:03:52.877178 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 20660
2026-06-20T23:03:52Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2743 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:03:52 dist-test-slave-f63b krb5kdc[21190](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:03:52 dist-test-slave-f63b krb5kdc[21190](info): set up 2 sockets
Jun 20 23:03:52 dist-test-slave-f63b krb5kdc[21190](info): commencing operation
krb5kdc: starting...
W20260620 23:03:53.990756 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.006s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 23:03:54 dist-test-slave-f63b krb5kdc[21190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:03:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:03:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
I20260620 23:03:54.241322 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:46113
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:35495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:46113
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:54.578990 21203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:54.579350 21203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:54.579510 21203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:54.586856 21203 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:03:54.587018 21203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:54.587122 21203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:54.587236 21203 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:03:54.587312 21203 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:03:54.598101 21203 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:35495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:46113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:46113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:03:54.599557 21203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:54.600898 21203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:54.613176 21208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:54.613271 21209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:54.613267 21211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:54.614234 21203 server_base.cc:1034] running on GCE node
I20260620 23:03:54.615213 21203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:54.616734 21203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:54.617971 21203 hybrid_clock.cc:648] HybridClock initialized: now 1781996634617922 us; error 44 us; skew 500 ppm
I20260620 23:03:54.618323 21203 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:03:54 dist-test-slave-f63b krb5kdc[21190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996634, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:54.621579 21203 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 23:03:54.623296 21203 webserver.cc:460] Webserver started at http://127.2.237.62:34699/ using document root <none> and password file <none>
I20260620 23:03:54.623986 21203 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:54.624135 21203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:54.624444 21203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:54.627517 21203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "fc3ceac17eb84d778d7b34c763492298"
format_stamp: "Formatted at 2026-06-20 23:03:54 on dist-test-slave-f63b"
server_key: "ff770e6abf883f2240aef9864aaefbbc"
server_key_iv: "d7d648ddfceac9d09f4c187ecfd43b15"
server_key_version: "encryptionkey@0"
I20260620 23:03:54.628302 21203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "fc3ceac17eb84d778d7b34c763492298"
format_stamp: "Formatted at 2026-06-20 23:03:54 on dist-test-slave-f63b"
server_key: "ff770e6abf883f2240aef9864aaefbbc"
server_key_iv: "d7d648ddfceac9d09f4c187ecfd43b15"
server_key_version: "encryptionkey@0"
I20260620 23:03:54.633426 21203 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260620 23:03:54.636976 21218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:54.638274 21203 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260620 23:03:54.638468 21203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "fc3ceac17eb84d778d7b34c763492298"
format_stamp: "Formatted at 2026-06-20 23:03:54 on dist-test-slave-f63b"
server_key: "ff770e6abf883f2240aef9864aaefbbc"
server_key_iv: "d7d648ddfceac9d09f4c187ecfd43b15"
server_key_version: "encryptionkey@0"
I20260620 23:03:54.638695 21203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:54.666163 21203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:54.670696 21203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:54.671015 21203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:54.690510 21203 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:46113
I20260620 23:03:54.690539 21269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:46113 every 8 connection(s)
I20260620 23:03:54.692073 21203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:03:54.696266 21270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:54.696480 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 21203
I20260620 23:03:54.696681 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:03:54.697322 2996 external_mini_cluster.cc:1428] Setting key d55d244095a215086a84d3ac6084d196
I20260620 23:03:54.713871 21270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc3ceac17eb84d778d7b34c763492298: Bootstrap starting.
Jun 20 23:03:54 dist-test-slave-f63b krb5kdc[21190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:03:54.717891 21270 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc3ceac17eb84d778d7b34c763492298: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:54.719156 21270 log.cc:824] T 00000000000000000000000000000000 P fc3ceac17eb84d778d7b34c763492298: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:54.722250 21270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc3ceac17eb84d778d7b34c763492298: No bootstrap required, opened a new log
I20260620 23:03:54.725095 21273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:54.699427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43130 (local address 127.2.237.62:46113)
0620 23:03:54.700402 (+ 975us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:54.700414 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:54.700457 (+ 43us) server_negotiation.cc:407] Connection header received
0620 23:03:54.704723 (+ 4266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:54.704761 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:54.705324 (+ 563us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:54.705821 (+ 497us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:54.707962 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:54.709239 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:54.711345 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:54.712075 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:54.712236 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:54.717409 (+ 5173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:54.717424 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:54.717440 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:54.717481 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:54.720783 (+ 3302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:54.721525 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:54.721532 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:54.721538 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:54.721710 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:54.722119 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:54.722122 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:54.722124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:54.722771 (+ 647us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:54.723097 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:54.723617 (+ 520us) server_negotiation.cc:299] Negotiation successful
0620 23:03:54.724047 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":758,"thread_start_us":529,"threads_started":1}
I20260620 23:03:54.731582 21270 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc3ceac17eb84d778d7b34c763492298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3ceac17eb84d778d7b34c763492298" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 46113 } }
I20260620 23:03:54.732024 21270 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc3ceac17eb84d778d7b34c763492298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:54.732139 21270 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc3ceac17eb84d778d7b34c763492298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc3ceac17eb84d778d7b34c763492298, State: Initialized, Role: FOLLOWER
I20260620 23:03:54.732789 21270 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc3ceac17eb84d778d7b34c763492298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3ceac17eb84d778d7b34c763492298" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 46113 } }
I20260620 23:03:54.733073 21270 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc3ceac17eb84d778d7b34c763492298 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:03:54.733232 21270 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc3ceac17eb84d778d7b34c763492298 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:03:54.733431 21270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc3ceac17eb84d778d7b34c763492298 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:54.735904 21270 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc3ceac17eb84d778d7b34c763492298 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3ceac17eb84d778d7b34c763492298" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 46113 } }
I20260620 23:03:54.736418 21270 leader_election.cc:302] T 00000000000000000000000000000000 P fc3ceac17eb84d778d7b34c763492298 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc3ceac17eb84d778d7b34c763492298; no voters:
I20260620 23:03:54.737071 21270 leader_election.cc:288] T 00000000000000000000000000000000 P fc3ceac17eb84d778d7b34c763492298 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:03:54.737263 21275 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc3ceac17eb84d778d7b34c763492298 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:54.738296 21275 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc3ceac17eb84d778d7b34c763492298 [term 1 LEADER]: Becoming Leader. State: Replica: fc3ceac17eb84d778d7b34c763492298, State: Running, Role: LEADER
I20260620 23:03:54.739018 21275 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc3ceac17eb84d778d7b34c763492298 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3ceac17eb84d778d7b34c763492298" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 46113 } }
I20260620 23:03:54.739485 21270 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc3ceac17eb84d778d7b34c763492298 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:03:54.742136 21277 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc3ceac17eb84d778d7b34c763492298 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc3ceac17eb84d778d7b34c763492298. Latest consensus state: current_term: 1 leader_uuid: "fc3ceac17eb84d778d7b34c763492298" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3ceac17eb84d778d7b34c763492298" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 46113 } } }
I20260620 23:03:54.742645 21277 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc3ceac17eb84d778d7b34c763492298 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:54.743192 21276 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc3ceac17eb84d778d7b34c763492298 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc3ceac17eb84d778d7b34c763492298" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3ceac17eb84d778d7b34c763492298" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 46113 } } }
I20260620 23:03:54.743497 21276 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc3ceac17eb84d778d7b34c763492298 [sys.catalog]: This master's current role is: LEADER
W20260620 23:03:54.752164 21291 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fc3ceac17eb84d778d7b34c763492298: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:03:54.752329 21291 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:03:54.752528 21283 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:03:54.754295 21283 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:03:54.763268 21283 catalog_manager.cc:1347] Generated new cluster ID: 446be4f395d7428fbab8f0249e3a9860
I20260620 23:03:54.763417 21283 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:03:54.780875 21283 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:03:54.782832 21283 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:03:54.796324 21283 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc3ceac17eb84d778d7b34c763492298: Generated new TSK 0
I20260620 23:03:54.797465 21283 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.237.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
I20260620 23:03:54.865896 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:0
--local_ip_for_outbound_sockets=127.2.237.1
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:46113
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:35495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:55.202948 21296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:55.203289 21296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:55.203454 21296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:55.211012 21296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:55.211174 21296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:55.211344 21296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 23:03:55.221352 21296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:35495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:46113
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:03:55.222777 21296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:55.224277 21296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:55.238227 21301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:55.238277 21302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:55.238310 21304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:55.238999 21296 server_base.cc:1034] running on GCE node
I20260620 23:03:55.239658 21296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:55.240751 21296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:55.241976 21296 hybrid_clock.cc:648] HybridClock initialized: now 1781996635241918 us; error 52 us; skew 500 ppm
I20260620 23:03:55.242332 21296 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:03:55 dist-test-slave-f63b krb5kdc[21190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996635, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:55.245954 21296 init.cc:377] Logged in from keytab as kudu/127.2.237.1@KRBTEST.COM (short username kudu)
I20260620 23:03:55.247476 21296 webserver.cc:460] Webserver started at http://127.2.237.1:39879/ using document root <none> and password file <none>
I20260620 23:03:55.248196 21296 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:55.248344 21296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:55.248675 21296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:55.251760 21296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-0/data/instance:
uuid: "5b0efb27351841338b16b33b478b8e3c"
format_stamp: "Formatted at 2026-06-20 23:03:55 on dist-test-slave-f63b"
server_key: "fade44446ca3ef1298f1faa2d1b3b91e"
server_key_iv: "c4488963e8e10b7c8a2e9ac643787ae1"
server_key_version: "encryptionkey@0"
I20260620 23:03:55.252569 21296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance:
uuid: "5b0efb27351841338b16b33b478b8e3c"
format_stamp: "Formatted at 2026-06-20 23:03:55 on dist-test-slave-f63b"
server_key: "fade44446ca3ef1298f1faa2d1b3b91e"
server_key_iv: "c4488963e8e10b7c8a2e9ac643787ae1"
server_key_version: "encryptionkey@0"
I20260620 23:03:55.258025 21296 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260620 23:03:55.262120 21311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:55.263727 21296 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 23:03:55.263968 21296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "5b0efb27351841338b16b33b478b8e3c"
format_stamp: "Formatted at 2026-06-20 23:03:55 on dist-test-slave-f63b"
server_key: "fade44446ca3ef1298f1faa2d1b3b91e"
server_key_iv: "c4488963e8e10b7c8a2e9ac643787ae1"
server_key_version: "encryptionkey@0"
I20260620 23:03:55.264209 21296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:55.279436 21296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:55.283716 21296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:55.284030 21296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:55.285028 21296 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:55.286442 21296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:55.286548 21296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:55.286711 21296 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:55.286792 21296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:55.320569 21296 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:42953
I20260620 23:03:55.320658 21423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:42953 every 8 connection(s)
I20260620 23:03:55.321945 21296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 23:03:55.323333 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 21296
I20260620 23:03:55.323503 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance
I20260620 23:03:55.324072 2996 external_mini_cluster.cc:1428] Setting key d0f46e6e4689c538b2dbd088fb999334
Jun 20 23:03:55 dist-test-slave-f63b krb5kdc[21190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996635, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:03:55.343303 21426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:55.324859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:59585 (local address 127.2.237.62:46113)
0620 23:03:55.325352 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:55.325356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:55.326657 (+ 1301us) server_negotiation.cc:407] Connection header received
0620 23:03:55.328313 (+ 1656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:55.328322 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:55.328426 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:55.328543 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:55.329994 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:55.331158 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:55.332782 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:55.333506 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:55.333647 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:55.337413 (+ 3766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:55.337418 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:55.337421 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:55.337460 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:55.340226 (+ 2766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:55.340963 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:55.340969 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:55.340973 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:55.341099 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:55.341520 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:55.341527 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:55.341531 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:55.341790 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:55.341945 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:55.342688 (+ 743us) server_negotiation.cc:299] Negotiation successful
0620 23:03:55.342934 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":258,"threads_started":1}
I20260620 23:03:55.344319 21427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:55.325353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:46113 (local address 127.2.237.1:59585)
0620 23:03:55.326205 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:55.326257 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:55.328037 (+ 1780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:55.328739 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:55.328748 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:55.329279 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:55.329824 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:55.329840 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:55.331348 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:55.331352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:55.332629 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:55.332641 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:55.333722 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:55.333727 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:55.333932 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:55.334715 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:55.334745 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:55.337183 (+ 2438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:55.340403 (+ 3220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:55.340411 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:55.340433 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:55.340839 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:55.341209 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:55.341215 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:55.341219 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:55.341409 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:55.341918 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:55.341926 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:55.342270 (+ 344us) client_negotiation.cc:769] Sending connection context
0620 23:03:55.342538 (+ 268us) client_negotiation.cc:240] Negotiation successful
0620 23:03:55.343169 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":362,"threads_started":1}
I20260620 23:03:55.345948 21424 heartbeater.cc:347] Connected to a master server at 127.2.237.62:46113
I20260620 23:03:55.346283 21424 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:55.347069 21424 heartbeater.cc:511] Master 127.2.237.62:46113 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.2.237.2@KRBTEST.COM; defaulting to no policy
I20260620 23:03:55.349319 21235 ts_manager.cc:194] Registered new tserver with Master: 5b0efb27351841338b16b33b478b8e3c (127.2.237.1:42953)
I20260620 23:03:55.350834 21235 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.1@KRBTEST.COM'} at 127.2.237.1:59585
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
I20260620 23:03:55.382256 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:0
--local_ip_for_outbound_sockets=127.2.237.2
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:46113
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:35495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:55.721930 21430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:55.722272 21430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:55.722468 21430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:55.729918 21430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:55.730082 21430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:55.730288 21430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 23:03:55.740543 21430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:35495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:46113
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:03:55.741992 21430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:55.743593 21430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:55.756971 21436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:55.757095 21438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:55.757130 21435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:55.758111 21430 server_base.cc:1034] running on GCE node
I20260620 23:03:55.758741 21430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:55.759617 21430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:55.760816 21430 hybrid_clock.cc:648] HybridClock initialized: now 1781996635760769 us; error 39 us; skew 500 ppm
I20260620 23:03:55.761140 21430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:03:55 dist-test-slave-f63b krb5kdc[21190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996635, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:55.764415 21430 init.cc:377] Logged in from keytab as kudu/127.2.237.2@KRBTEST.COM (short username kudu)
I20260620 23:03:55.765836 21430 webserver.cc:460] Webserver started at http://127.2.237.2:42805/ using document root <none> and password file <none>
I20260620 23:03:55.766530 21430 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:55.766757 21430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:55.767148 21430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:55.770310 21430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-1/data/instance:
uuid: "a4ee4c85505a448f8f21c7bf81241781"
format_stamp: "Formatted at 2026-06-20 23:03:55 on dist-test-slave-f63b"
server_key: "d84980b43c8622153761931f0f780e05"
server_key_iv: "06ae3a01cc30eb0f8f52a391c29d2d8e"
server_key_version: "encryptionkey@0"
I20260620 23:03:55.771224 21430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance:
uuid: "a4ee4c85505a448f8f21c7bf81241781"
format_stamp: "Formatted at 2026-06-20 23:03:55 on dist-test-slave-f63b"
server_key: "d84980b43c8622153761931f0f780e05"
server_key_iv: "06ae3a01cc30eb0f8f52a391c29d2d8e"
server_key_version: "encryptionkey@0"
I20260620 23:03:55.776407 21430 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260620 23:03:55.779975 21445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:55.781373 21430 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 23:03:55.781656 21430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "a4ee4c85505a448f8f21c7bf81241781"
format_stamp: "Formatted at 2026-06-20 23:03:55 on dist-test-slave-f63b"
server_key: "d84980b43c8622153761931f0f780e05"
server_key_iv: "06ae3a01cc30eb0f8f52a391c29d2d8e"
server_key_version: "encryptionkey@0"
I20260620 23:03:55.781847 21430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:55.799870 21430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:55.804338 21430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:55.804646 21430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:55.805624 21430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:55.807047 21430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:55.807118 21430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:55.807284 21430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:55.807334 21430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:55.840509 21430 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:44773
I20260620 23:03:55.840644 21557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:44773 every 8 connection(s)
I20260620 23:03:55.841938 21430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 23:03:55.849261 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 21430
I20260620 23:03:55.849442 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance
I20260620 23:03:55.849865 2996 external_mini_cluster.cc:1428] Setting key f263aa9e16ac083f1d4bb9352552242f
Jun 20 23:03:55 dist-test-slave-f63b krb5kdc[21190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996635, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:03:55.862365 21560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:55.844794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:43155 (local address 127.2.237.62:46113)
0620 23:03:55.845292 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:55.845297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:55.846312 (+ 1015us) server_negotiation.cc:407] Connection header received
0620 23:03:55.848262 (+ 1950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:55.848268 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:55.848370 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:55.848498 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:55.850095 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:55.850970 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:55.852680 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:55.853118 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:55.853258 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:55.856652 (+ 3394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:55.856657 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:55.856660 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:55.856690 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:55.859248 (+ 2558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:55.860094 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:55.860098 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:55.860100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:55.860216 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:55.860695 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:55.860699 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:55.860700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:55.860992 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:55.861104 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:55.861919 (+ 815us) server_negotiation.cc:299] Negotiation successful
0620 23:03:55.862108 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":257,"threads_started":1}
I20260620 23:03:55.863847 21561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:55.845441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:46113 (local address 127.2.237.2:43155)
0620 23:03:55.846147 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:55.846194 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:55.847994 (+ 1800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:55.848726 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:55.848734 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:55.849246 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:55.849918 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:55.849946 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:55.851132 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:55.851136 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:55.852535 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:55.852546 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:55.853353 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:55.853360 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:55.853581 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:55.854255 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:55.854281 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:55.856429 (+ 2148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:55.859381 (+ 2952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:55.859391 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:55.859407 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:55.859975 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:55.860336 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:55.860343 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:55.860349 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:55.860590 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:55.861113 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:55.861123 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:55.861558 (+ 435us) client_negotiation.cc:769] Sending connection context
0620 23:03:55.861848 (+ 290us) client_negotiation.cc:240] Negotiation successful
0620 23:03:55.862422 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":266,"threads_started":1}
I20260620 23:03:55.865255 21558 heartbeater.cc:347] Connected to a master server at 127.2.237.62:46113
I20260620 23:03:55.865556 21558 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:55.866217 21558 heartbeater.cc:511] Master 127.2.237.62:46113 requested a full tablet report, sending...
I20260620 23:03:55.867707 21235 ts_manager.cc:194] Registered new tserver with Master: a4ee4c85505a448f8f21c7bf81241781 (127.2.237.2:44773)
I20260620 23:03:55.868603 21235 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.2@KRBTEST.COM'} at 127.2.237.2:43155
WARNING: no policy specified for kudu/127.2.237.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
I20260620 23:03:55.903878 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:0
--local_ip_for_outbound_sockets=127.2.237.3
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:46113
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:35495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:03:56.243978 21564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:56.244331 21564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:56.244520 21564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:56.252693 21564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:56.252847 21564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:56.253046 21564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 23:03:56.262859 21564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:35495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:46113
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:03:56.264170 21564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:56.265539 21564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:56.279150 21569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:56.279294 21572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:56.279400 21570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:56.279860 21564 server_base.cc:1034] running on GCE node
I20260620 23:03:56.280489 21564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:56.281394 21564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:56.282678 21564 hybrid_clock.cc:648] HybridClock initialized: now 1781996636282631 us; error 58 us; skew 500 ppm
I20260620 23:03:56.282963 21564 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:03:56 dist-test-slave-f63b krb5kdc[21190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996636, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:03:56.286707 21564 init.cc:377] Logged in from keytab as kudu/127.2.237.3@KRBTEST.COM (short username kudu)
I20260620 23:03:56.288309 21564 webserver.cc:460] Webserver started at http://127.2.237.3:44959/ using document root <none> and password file <none>
I20260620 23:03:56.288959 21564 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:56.289055 21564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:56.289304 21564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:56.292130 21564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-2/data/instance:
uuid: "fdf4f5dd9966480989eb091f8573a141"
format_stamp: "Formatted at 2026-06-20 23:03:56 on dist-test-slave-f63b"
server_key: "84d17082c8beffc616ab046bacea7acc"
server_key_iv: "874ff31efe9ff1e33b14d579d92167e2"
server_key_version: "encryptionkey@0"
I20260620 23:03:56.292850 21564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance:
uuid: "fdf4f5dd9966480989eb091f8573a141"
format_stamp: "Formatted at 2026-06-20 23:03:56 on dist-test-slave-f63b"
server_key: "84d17082c8beffc616ab046bacea7acc"
server_key_iv: "874ff31efe9ff1e33b14d579d92167e2"
server_key_version: "encryptionkey@0"
I20260620 23:03:56.298019 21564 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260620 23:03:56.301818 21579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:56.303349 21564 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 23:03:56.303514 21564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "fdf4f5dd9966480989eb091f8573a141"
format_stamp: "Formatted at 2026-06-20 23:03:56 on dist-test-slave-f63b"
server_key: "84d17082c8beffc616ab046bacea7acc"
server_key_iv: "874ff31efe9ff1e33b14d579d92167e2"
server_key_version: "encryptionkey@0"
I20260620 23:03:56.303692 21564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:56.324374 21564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:56.329277 21564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:56.329594 21564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:56.330680 21564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:56.332134 21564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:56.332247 21564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:56.332420 21564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:56.332502 21564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:56.353847 21424 heartbeater.cc:503] Master 127.2.237.62:46113 was elected leader, sending a full tablet report...
I20260620 23:03:56.370741 21564 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:41137
I20260620 23:03:56.370798 21691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:41137 every 8 connection(s)
I20260620 23:03:56.372253 21564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 23:03:56.378417 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 21564
I20260620 23:03:56.378659 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance
I20260620 23:03:56.379356 2996 external_mini_cluster.cc:1428] Setting key aefb5aa8e294d5ec3c812e4186c050e6
Jun 20 23:03:56 dist-test-slave-f63b krb5kdc[21190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996636, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:03:56.394380 21694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:56.374937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:56047 (local address 127.2.237.62:46113)
0620 23:03:56.375387 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:56.375393 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:56.376528 (+ 1135us) server_negotiation.cc:407] Connection header received
0620 23:03:56.378877 (+ 2349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:56.378883 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:56.378969 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:56.379082 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:56.381241 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.382112 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.383920 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.384323 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.384444 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:56.388261 (+ 3817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:56.388266 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:56.388269 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:56.388305 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:56.391055 (+ 2750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:56.391785 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:56.391789 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:56.391791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:56.391910 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:56.392328 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:56.392331 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:56.392333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:56.392593 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:56.392730 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:56.393691 (+ 961us) server_negotiation.cc:299] Negotiation successful
0620 23:03:56.393990 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":216,"threads_started":1}
I20260620 23:03:56.395541 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:56.375549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:46113 (local address 127.2.237.3:56047)
0620 23:03:56.376323 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:56.376392 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:56.378509 (+ 2117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:56.379410 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:56.379425 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:56.380145 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:56.381012 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:56.381043 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.382285 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.382299 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:56.383770 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:56.383783 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.384577 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.384582 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:56.384872 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:56.385581 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:03:56.385609 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:56.388044 (+ 2435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:56.391186 (+ 3142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:56.391194 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:56.391207 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:56.391656 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:56.392044 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:03:56.392048 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:03:56.392050 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:03:56.392229 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:03:56.392702 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:56.392708 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:56.393206 (+ 498us) client_negotiation.cc:769] Sending connection context
0620 23:03:56.393551 (+ 345us) client_negotiation.cc:240] Negotiation successful
0620 23:03:56.394103 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":299,"threads_started":1}
I20260620 23:03:56.396939 21692 heartbeater.cc:347] Connected to a master server at 127.2.237.62:46113
I20260620 23:03:56.397257 21692 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:56.397981 21692 heartbeater.cc:511] Master 127.2.237.62:46113 requested a full tablet report, sending...
I20260620 23:03:56.399384 21235 ts_manager.cc:194] Registered new tserver with Master: fdf4f5dd9966480989eb091f8573a141 (127.2.237.3:41137)
I20260620 23:03:56.400141 21235 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.3@KRBTEST.COM'} at 127.2.237.3:56047
I20260620 23:03:56.407552 2996 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:03:56.424959 21694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:56.411962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43132 (local address 127.2.237.62:46113)
0620 23:03:56.412124 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:56.412129 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:56.412617 (+ 488us) server_negotiation.cc:407] Connection header received
0620 23:03:56.412769 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:56.412773 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:56.412851 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:56.412938 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:56.414294 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.415121 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.416497 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.416901 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.417002 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:56.419627 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:56.419632 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:56.419635 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:56.419666 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:56.422279 (+ 2613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:56.423201 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:56.423206 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:56.423209 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:56.423342 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:56.423753 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:56.423756 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:56.423758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:56.424046 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:56.424239 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:56.424599 (+ 360us) server_negotiation.cc:299] Negotiation successful
0620 23:03:56.424736 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 23:03:56.429160 21235 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43132:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:03:56.436071 21235 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:03:56.471810 21707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:56.448869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57082 (local address 127.2.237.2:44773)
0620 23:03:56.462865 (+ 13996us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:56.462879 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:56.462915 (+ 36us) server_negotiation.cc:407] Connection header received
0620 23:03:56.463039 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:56.463049 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:56.463349 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:56.463572 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:56.464312 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.465803 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.469127 (+ 3324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.470612 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.470750 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:56.470976 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:56.471294 (+ 318us) server_negotiation.cc:299] Negotiation successful
0620 23:03:56.471478 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13722,"spinlock_wait_cycles":28544,"thread_start_us":13673,"threads_started":1}
I20260620 23:03:56.471884 21705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:56.448675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:44773 (local address 127.0.0.1:57082)
0620 23:03:56.462064 (+ 13389us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:56.462126 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:56.462346 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:56.463626 (+ 1280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:56.463637 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:56.463686 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:56.464142 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:56.464155 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.465970 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.465975 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:56.468939 (+ 2964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:56.468955 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.470812 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.470819 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:56.471032 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:56.471081 (+ 49us) client_negotiation.cc:769] Sending connection context
0620 23:03:56.471234 (+ 153us) client_negotiation.cc:240] Negotiation successful
0620 23:03:56.471406 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13033,"mutex_wait_us":1,"thread_start_us":12842,"threads_started":1}
I20260620 23:03:56.474040 21706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:56.448679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:41137 (local address 127.0.0.1:35962)
0620 23:03:56.463372 (+ 14693us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:56.463392 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:56.463514 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:56.465118 (+ 1604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:56.465122 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:56.465148 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:56.466966 (+ 1818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:56.466981 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.468545 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.468548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:56.470723 (+ 2175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:56.470737 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.473218 (+ 2481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.473224 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:56.473375 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:56.473404 (+ 29us) client_negotiation.cc:769] Sending connection context
0620 23:03:56.473489 (+ 85us) client_negotiation.cc:240] Negotiation successful
0620 23:03:56.473611 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14588,"mutex_wait_us":25,"thread_start_us":14886,"threads_started":1}
I20260620 23:03:56.474788 21708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:56.448822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35962 (local address 127.2.237.3:41137)
0620 23:03:56.464273 (+ 15451us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:56.464282 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:56.464315 (+ 33us) server_negotiation.cc:407] Connection header received
0620 23:03:56.464411 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:56.464427 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:56.464668 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:56.464867 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:56.467214 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.468333 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.470894 (+ 2561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.472626 (+ 1732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.472756 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:56.473044 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:56.474349 (+ 1305us) server_negotiation.cc:299] Negotiation successful
0620 23:03:56.474474 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15261,"thread_start_us":15084,"threads_started":1}
I20260620 23:03:56.476459 21493 tablet_service.cc:1467] Processing CreateTablet for tablet 5fa085c5c28b466394b213cef6ccdaec (DEFAULT_TABLE table=test-table [id=bfb313a1f8c44a01b18e0f1ce07dc82b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:56.477986 21493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fa085c5c28b466394b213cef6ccdaec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:56.478214 21710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:56.465087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:42953 (local address 127.0.0.1:55206)
0620 23:03:56.465652 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:56.465668 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:56.465874 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:56.466511 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:56.466514 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:56.466536 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:56.468011 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:56.468023 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.469759 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.469762 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:56.471638 (+ 1876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:56.471653 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.477367 (+ 5714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.477372 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:56.477496 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:56.477517 (+ 21us) client_negotiation.cc:769] Sending connection context
0620 23:03:56.477621 (+ 104us) client_negotiation.cc:240] Negotiation successful
0620 23:03:56.477745 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"mutex_wait_us":28,"thread_start_us":323,"threads_started":1}
I20260620 23:03:56.478216 21709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:56.464413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55206 (local address 127.2.237.1:42953)
0620 23:03:56.465085 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:56.465093 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:56.465757 (+ 664us) server_negotiation.cc:407] Connection header received
0620 23:03:56.466105 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:56.466113 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:56.466327 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:56.466508 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:56.468285 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.469594 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.471830 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.473213 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.473318 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:56.473524 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:56.477678 (+ 4154us) server_negotiation.cc:299] Negotiation successful
0620 23:03:56.477849 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":351,"threads_started":1}
I20260620 23:03:56.478257 21627 tablet_service.cc:1467] Processing CreateTablet for tablet 5fa085c5c28b466394b213cef6ccdaec (DEFAULT_TABLE table=test-table [id=bfb313a1f8c44a01b18e0f1ce07dc82b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:56.479745 21627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fa085c5c28b466394b213cef6ccdaec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:56.481798 21359 tablet_service.cc:1467] Processing CreateTablet for tablet 5fa085c5c28b466394b213cef6ccdaec (DEFAULT_TABLE table=test-table [id=bfb313a1f8c44a01b18e0f1ce07dc82b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:56.482928 21359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fa085c5c28b466394b213cef6ccdaec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:56.492954 21711 tablet_bootstrap.cc:492] T 5fa085c5c28b466394b213cef6ccdaec P a4ee4c85505a448f8f21c7bf81241781: Bootstrap starting.
I20260620 23:03:56.495208 21712 tablet_bootstrap.cc:492] T 5fa085c5c28b466394b213cef6ccdaec P 5b0efb27351841338b16b33b478b8e3c: Bootstrap starting.
I20260620 23:03:56.496989 21711 tablet_bootstrap.cc:654] T 5fa085c5c28b466394b213cef6ccdaec P a4ee4c85505a448f8f21c7bf81241781: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:56.497992 21711 log.cc:824] T 5fa085c5c28b466394b213cef6ccdaec P a4ee4c85505a448f8f21c7bf81241781: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:56.499442 21713 tablet_bootstrap.cc:492] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141: Bootstrap starting.
I20260620 23:03:56.499454 21712 tablet_bootstrap.cc:654] T 5fa085c5c28b466394b213cef6ccdaec P 5b0efb27351841338b16b33b478b8e3c: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:56.500607 21712 log.cc:824] T 5fa085c5c28b466394b213cef6ccdaec P 5b0efb27351841338b16b33b478b8e3c: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:56.502633 21713 tablet_bootstrap.cc:654] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:56.503376 21712 tablet_bootstrap.cc:492] T 5fa085c5c28b466394b213cef6ccdaec P 5b0efb27351841338b16b33b478b8e3c: No bootstrap required, opened a new log
I20260620 23:03:56.503593 21713 log.cc:824] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:56.503633 21712 ts_tablet_manager.cc:1397] T 5fa085c5c28b466394b213cef6ccdaec P 5b0efb27351841338b16b33b478b8e3c: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260620 23:03:56.505240 21711 tablet_bootstrap.cc:492] T 5fa085c5c28b466394b213cef6ccdaec P a4ee4c85505a448f8f21c7bf81241781: No bootstrap required, opened a new log
I20260620 23:03:56.505512 21711 ts_tablet_manager.cc:1397] T 5fa085c5c28b466394b213cef6ccdaec P a4ee4c85505a448f8f21c7bf81241781: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.004s
I20260620 23:03:56.507869 21713 tablet_bootstrap.cc:492] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141: No bootstrap required, opened a new log
I20260620 23:03:56.508172 21713 ts_tablet_manager.cc:1397] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.005s
I20260620 23:03:56.511720 21712 raft_consensus.cc:348] T 5fa085c5c28b466394b213cef6ccdaec P 5b0efb27351841338b16b33b478b8e3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdf4f5dd9966480989eb091f8573a141" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 41137 } } peers { permanent_uuid: "a4ee4c85505a448f8f21c7bf81241781" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 44773 } } peers { permanent_uuid: "5b0efb27351841338b16b33b478b8e3c" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42953 } }
I20260620 23:03:56.512089 21712 raft_consensus.cc:374] T 5fa085c5c28b466394b213cef6ccdaec P 5b0efb27351841338b16b33b478b8e3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:56.512212 21712 raft_consensus.cc:729] T 5fa085c5c28b466394b213cef6ccdaec P 5b0efb27351841338b16b33b478b8e3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b0efb27351841338b16b33b478b8e3c, State: Initialized, Role: FOLLOWER
I20260620 23:03:56.512807 21712 consensus_queue.cc:260] T 5fa085c5c28b466394b213cef6ccdaec P 5b0efb27351841338b16b33b478b8e3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdf4f5dd9966480989eb091f8573a141" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 41137 } } peers { permanent_uuid: "a4ee4c85505a448f8f21c7bf81241781" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 44773 } } peers { permanent_uuid: "5b0efb27351841338b16b33b478b8e3c" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42953 } }
I20260620 23:03:56.516023 21711 raft_consensus.cc:348] T 5fa085c5c28b466394b213cef6ccdaec P a4ee4c85505a448f8f21c7bf81241781 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdf4f5dd9966480989eb091f8573a141" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 41137 } } peers { permanent_uuid: "a4ee4c85505a448f8f21c7bf81241781" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 44773 } } peers { permanent_uuid: "5b0efb27351841338b16b33b478b8e3c" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42953 } }
I20260620 23:03:56.516489 21711 raft_consensus.cc:374] T 5fa085c5c28b466394b213cef6ccdaec P a4ee4c85505a448f8f21c7bf81241781 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:56.516598 21711 raft_consensus.cc:729] T 5fa085c5c28b466394b213cef6ccdaec P a4ee4c85505a448f8f21c7bf81241781 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4ee4c85505a448f8f21c7bf81241781, State: Initialized, Role: FOLLOWER
I20260620 23:03:56.517184 21711 consensus_queue.cc:260] T 5fa085c5c28b466394b213cef6ccdaec P a4ee4c85505a448f8f21c7bf81241781 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdf4f5dd9966480989eb091f8573a141" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 41137 } } peers { permanent_uuid: "a4ee4c85505a448f8f21c7bf81241781" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 44773 } } peers { permanent_uuid: "5b0efb27351841338b16b33b478b8e3c" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42953 } }
I20260620 23:03:56.518491 21558 heartbeater.cc:503] Master 127.2.237.62:46113 was elected leader, sending a full tablet report...
I20260620 23:03:56.518726 21713 raft_consensus.cc:348] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdf4f5dd9966480989eb091f8573a141" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 41137 } } peers { permanent_uuid: "a4ee4c85505a448f8f21c7bf81241781" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 44773 } } peers { permanent_uuid: "5b0efb27351841338b16b33b478b8e3c" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42953 } }
I20260620 23:03:56.519132 21713 raft_consensus.cc:374] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:56.519227 21713 raft_consensus.cc:729] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdf4f5dd9966480989eb091f8573a141, State: Initialized, Role: FOLLOWER
I20260620 23:03:56.519767 21713 consensus_queue.cc:260] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdf4f5dd9966480989eb091f8573a141" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 41137 } } peers { permanent_uuid: "a4ee4c85505a448f8f21c7bf81241781" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 44773 } } peers { permanent_uuid: "5b0efb27351841338b16b33b478b8e3c" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42953 } }
I20260620 23:03:56.520860 21711 ts_tablet_manager.cc:1428] T 5fa085c5c28b466394b213cef6ccdaec P a4ee4c85505a448f8f21c7bf81241781: Time spent starting tablet: real 0.015s user 0.008s sys 0.004s
I20260620 23:03:56.521813 21692 heartbeater.cc:503] Master 127.2.237.62:46113 was elected leader, sending a full tablet report...
I20260620 23:03:56.522913 21712 ts_tablet_manager.cc:1428] T 5fa085c5c28b466394b213cef6ccdaec P 5b0efb27351841338b16b33b478b8e3c: Time spent starting tablet: real 0.019s user 0.014s sys 0.003s
I20260620 23:03:56.523749 21713 ts_tablet_manager.cc:1428] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260620 23:03:56.571859 21719 raft_consensus.cc:482] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:03:56.572142 21719 raft_consensus.cc:504] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdf4f5dd9966480989eb091f8573a141" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 41137 } } peers { permanent_uuid: "a4ee4c85505a448f8f21c7bf81241781" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 44773 } } peers { permanent_uuid: "5b0efb27351841338b16b33b478b8e3c" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42953 } }
I20260620 23:03:56.574043 21719 leader_election.cc:288] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4ee4c85505a448f8f21c7bf81241781 (127.2.237.2:44773), 5b0efb27351841338b16b33b478b8e3c (127.2.237.1:42953)
W20260620 23:03:56.574762 21425 tablet.cc:2369] T 5fa085c5c28b466394b213cef6ccdaec P 5b0efb27351841338b16b33b478b8e3c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:03:56.580132 21707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:56.574514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:60157 (local address 127.2.237.2:44773)
0620 23:03:56.574778 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:56.574788 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:56.575084 (+ 296us) server_negotiation.cc:407] Connection header received
0620 23:03:56.575170 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:56.575175 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:56.575254 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:56.575383 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:56.576041 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.576746 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.578321 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.579236 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.579396 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:56.579491 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:56.579746 (+ 255us) server_negotiation.cc:299] Negotiation successful
0620 23:03:56.579868 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260620 23:03:56.580129 21695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:56.574234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:44773 (local address 127.2.237.3:60157)
0620 23:03:56.574899 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:56.574922 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:56.575059 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:56.575604 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:56.575612 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:56.575644 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:56.575866 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:56.575888 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.576908 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.576914 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:56.578158 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:56.578172 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.579397 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.579404 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:56.579592 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:56.579616 (+ 24us) client_negotiation.cc:769] Sending connection context
0620 23:03:56.579722 (+ 106us) client_negotiation.cc:240] Negotiation successful
0620 23:03:56.579852 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509}
I20260620 23:03:56.581077 21709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:56.575374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:54307 (local address 127.2.237.1:42953)
0620 23:03:56.575817 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:56.575822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:56.576029 (+ 207us) server_negotiation.cc:407] Connection header received
0620 23:03:56.576223 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:56.576227 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:56.576283 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:56.576378 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:56.576900 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.577605 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.579328 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.580092 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.580252 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:56.580347 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:56.580666 (+ 319us) server_negotiation.cc:299] Negotiation successful
0620 23:03:56.580822 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304}
I20260620 23:03:56.581103 21720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:56.574387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:42953 (local address 127.2.237.3:54307)
0620 23:03:56.575935 (+ 1548us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:56.575948 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:56.576091 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:56.576403 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:56.576408 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:56.576440 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:56.576754 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:56.576761 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.577750 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.577756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:56.579185 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:56.579196 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.580287 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.580293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:56.580495 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:56.580537 (+ 42us) client_negotiation.cc:769] Sending connection context
0620 23:03:56.580630 (+ 93us) client_negotiation.cc:240] Negotiation successful
0620 23:03:56.580760 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1437,"thread_start_us":1186,"threads_started":1}
I20260620 23:03:56.581326 21513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fa085c5c28b466394b213cef6ccdaec" candidate_uuid: "fdf4f5dd9966480989eb091f8573a141" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4ee4c85505a448f8f21c7bf81241781" is_pre_election: true
I20260620 23:03:56.581957 21513 raft_consensus.cc:2454] T 5fa085c5c28b466394b213cef6ccdaec P a4ee4c85505a448f8f21c7bf81241781 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdf4f5dd9966480989eb091f8573a141 in term 0.
I20260620 23:03:56.582178 21379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fa085c5c28b466394b213cef6ccdaec" candidate_uuid: "fdf4f5dd9966480989eb091f8573a141" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b0efb27351841338b16b33b478b8e3c" is_pre_election: true
I20260620 23:03:56.582716 21583 leader_election.cc:302] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4ee4c85505a448f8f21c7bf81241781, fdf4f5dd9966480989eb091f8573a141; no voters:
I20260620 23:03:56.582782 21379 raft_consensus.cc:2454] T 5fa085c5c28b466394b213cef6ccdaec P 5b0efb27351841338b16b33b478b8e3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdf4f5dd9966480989eb091f8573a141 in term 0.
I20260620 23:03:56.583165 21719 raft_consensus.cc:2781] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:03:56.583333 21719 raft_consensus.cc:482] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:03:56.583472 21719 raft_consensus.cc:3037] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:56.585929 21719 raft_consensus.cc:504] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdf4f5dd9966480989eb091f8573a141" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 41137 } } peers { permanent_uuid: "a4ee4c85505a448f8f21c7bf81241781" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 44773 } } peers { permanent_uuid: "5b0efb27351841338b16b33b478b8e3c" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42953 } }
I20260620 23:03:56.586611 21719 leader_election.cc:288] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141 [CANDIDATE]: Term 1 election: Requested vote from peers a4ee4c85505a448f8f21c7bf81241781 (127.2.237.2:44773), 5b0efb27351841338b16b33b478b8e3c (127.2.237.1:42953)
I20260620 23:03:56.587033 21513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fa085c5c28b466394b213cef6ccdaec" candidate_uuid: "fdf4f5dd9966480989eb091f8573a141" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4ee4c85505a448f8f21c7bf81241781"
I20260620 23:03:56.587095 21379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fa085c5c28b466394b213cef6ccdaec" candidate_uuid: "fdf4f5dd9966480989eb091f8573a141" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b0efb27351841338b16b33b478b8e3c"
I20260620 23:03:56.587213 21513 raft_consensus.cc:3037] T 5fa085c5c28b466394b213cef6ccdaec P a4ee4c85505a448f8f21c7bf81241781 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:56.587337 21379 raft_consensus.cc:3037] T 5fa085c5c28b466394b213cef6ccdaec P 5b0efb27351841338b16b33b478b8e3c [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:56.589484 21379 raft_consensus.cc:2454] T 5fa085c5c28b466394b213cef6ccdaec P 5b0efb27351841338b16b33b478b8e3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdf4f5dd9966480989eb091f8573a141 in term 1.
I20260620 23:03:56.589529 21513 raft_consensus.cc:2454] T 5fa085c5c28b466394b213cef6ccdaec P a4ee4c85505a448f8f21c7bf81241781 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdf4f5dd9966480989eb091f8573a141 in term 1.
I20260620 23:03:56.589898 21581 leader_election.cc:302] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b0efb27351841338b16b33b478b8e3c, fdf4f5dd9966480989eb091f8573a141; no voters:
I20260620 23:03:56.590247 21719 raft_consensus.cc:2781] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:56.590719 21719 raft_consensus.cc:686] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141 [term 1 LEADER]: Becoming Leader. State: Replica: fdf4f5dd9966480989eb091f8573a141, State: Running, Role: LEADER
I20260620 23:03:56.591189 21719 consensus_queue.cc:237] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdf4f5dd9966480989eb091f8573a141" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 41137 } } peers { permanent_uuid: "a4ee4c85505a448f8f21c7bf81241781" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 44773 } } peers { permanent_uuid: "5b0efb27351841338b16b33b478b8e3c" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42953 } }
W20260620 23:03:56.594094 21559 tablet.cc:2369] T 5fa085c5c28b466394b213cef6ccdaec P a4ee4c85505a448f8f21c7bf81241781: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:03:56.595721 21233 catalog_manager.cc:5495] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141 reported cstate change: term changed from 0 to 1, leader changed from <none> to fdf4f5dd9966480989eb091f8573a141 (127.2.237.3). New cstate: current_term: 1 leader_uuid: "fdf4f5dd9966480989eb091f8573a141" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdf4f5dd9966480989eb091f8573a141" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 41137 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4ee4c85505a448f8f21c7bf81241781" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 44773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b0efb27351841338b16b33b478b8e3c" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42953 } health_report { overall_health: UNKNOWN } } }
I20260620 23:03:56.618095 21708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:56.612397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35978 (local address 127.2.237.3:41137)
0620 23:03:56.612625 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:56.612630 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:56.612649 (+ 19us) server_negotiation.cc:407] Connection header received
0620 23:03:56.612822 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:56.612826 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:56.612906 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:56.613072 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:03:56.613450 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.614218 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.615839 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.616240 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.616336 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:56.616836 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:03:56.617354 (+ 518us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:03:56.617494 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:56.617678 (+ 184us) server_negotiation.cc:299] Negotiation successful
0620 23:03:56.617747 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260620 23:03:56.624143 21693 tablet.cc:2369] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:03:56.626307 21513 raft_consensus.cc:1261] T 5fa085c5c28b466394b213cef6ccdaec P a4ee4c85505a448f8f21c7bf81241781 [term 1 FOLLOWER]: Refusing update from remote peer fdf4f5dd9966480989eb091f8573a141: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:56.626415 21379 raft_consensus.cc:1261] T 5fa085c5c28b466394b213cef6ccdaec P 5b0efb27351841338b16b33b478b8e3c [term 1 FOLLOWER]: Refusing update from remote peer fdf4f5dd9966480989eb091f8573a141: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:56.627311 21721 consensus_queue.cc:1035] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4ee4c85505a448f8f21c7bf81241781" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 44773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:03:56.627660 21719 consensus_queue.cc:1035] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b0efb27351841338b16b33b478b8e3c" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 42953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:03:56.640014 21724 mvcc.cc:205] Tried to move back new op lower bound from 7299058223611219968 to 7299058223481266176. Current Snapshot: MvccSnapshot[applied={T|T < 7299058223611219968}]
I20260620 23:03:56.653462 21233 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:03:56.653745 21233 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:03:56.657671 21233 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc3ceac17eb84d778d7b34c763492298: Sending DeleteTablet for 3 replicas of tablet 5fa085c5c28b466394b213cef6ccdaec
I20260620 23:03:56.658972 21359 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fa085c5c28b466394b213cef6ccdaec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:56 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:55206
I20260620 23:03:56.658994 21627 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fa085c5c28b466394b213cef6ccdaec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:56 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:35962
I20260620 23:03:56.658988 21493 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fa085c5c28b466394b213cef6ccdaec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:56 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:57082
I20260620 23:03:56.668782 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 21296
I20260620 23:03:56.671897 21734 tablet_replica.cc:330] T 5fa085c5c28b466394b213cef6ccdaec P a4ee4c85505a448f8f21c7bf81241781: stopping tablet replica
I20260620 23:03:56.672180 21733 tablet_replica.cc:330] T 5fa085c5c28b466394b213cef6ccdaec P 5b0efb27351841338b16b33b478b8e3c: stopping tablet replica
I20260620 23:03:56.672978 21733 raft_consensus.cc:2229] T 5fa085c5c28b466394b213cef6ccdaec P 5b0efb27351841338b16b33b478b8e3c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:03:56.673537 21734 raft_consensus.cc:2229] T 5fa085c5c28b466394b213cef6ccdaec P a4ee4c85505a448f8f21c7bf81241781 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:03:56.673664 21733 raft_consensus.cc:2258] T 5fa085c5c28b466394b213cef6ccdaec P 5b0efb27351841338b16b33b478b8e3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:56.674429 21734 raft_consensus.cc:2258] T 5fa085c5c28b466394b213cef6ccdaec P a4ee4c85505a448f8f21c7bf81241781 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:56.674683 21735 tablet_replica.cc:330] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141: stopping tablet replica
I20260620 23:03:56.675192 21735 raft_consensus.cc:2229] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141 [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:03:56.675925 21735 raft_consensus.cc:2258] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:56.676736 21733 ts_tablet_manager.cc:1905] T 5fa085c5c28b466394b213cef6ccdaec P 5b0efb27351841338b16b33b478b8e3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:56.678947 21735 ts_tablet_manager.cc:1905] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:56.679507 21734 ts_tablet_manager.cc:1905] T 5fa085c5c28b466394b213cef6ccdaec P a4ee4c85505a448f8f21c7bf81241781: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:56.684973 21733 ts_tablet_manager.cc:1918] T 5fa085c5c28b466394b213cef6ccdaec P 5b0efb27351841338b16b33b478b8e3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:03:56.685125 21733 log.cc:1196] T 5fa085c5c28b466394b213cef6ccdaec P 5b0efb27351841338b16b33b478b8e3c: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-0/wal/wals/5fa085c5c28b466394b213cef6ccdaec
I20260620 23:03:56.685760 21733 ts_tablet_manager.cc:1939] T 5fa085c5c28b466394b213cef6ccdaec P 5b0efb27351841338b16b33b478b8e3c: Deleting consensus metadata
I20260620 23:03:56.687822 21220 catalog_manager.cc:4841] TS 5b0efb27351841338b16b33b478b8e3c (127.2.237.1:42953): tablet 5fa085c5c28b466394b213cef6ccdaec (table test-table [id=bfb313a1f8c44a01b18e0f1ce07dc82b]) successfully deleted
I20260620 23:03:56.690419 21734 ts_tablet_manager.cc:1918] T 5fa085c5c28b466394b213cef6ccdaec P a4ee4c85505a448f8f21c7bf81241781: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:03:56.690644 21734 log.cc:1196] T 5fa085c5c28b466394b213cef6ccdaec P a4ee4c85505a448f8f21c7bf81241781: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-1/wal/wals/5fa085c5c28b466394b213cef6ccdaec
I20260620 23:03:56.691134 21734 ts_tablet_manager.cc:1939] T 5fa085c5c28b466394b213cef6ccdaec P a4ee4c85505a448f8f21c7bf81241781: Deleting consensus metadata
I20260620 23:03:56.692801 21222 catalog_manager.cc:4841] TS a4ee4c85505a448f8f21c7bf81241781 (127.2.237.2:44773): tablet 5fa085c5c28b466394b213cef6ccdaec (table test-table [id=bfb313a1f8c44a01b18e0f1ce07dc82b]) successfully deleted
I20260620 23:03:56.693919 21735 ts_tablet_manager.cc:1918] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:03:56.694159 21735 log.cc:1196] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781996318807807-2996-0/minicluster-data/ts-2/wal/wals/5fa085c5c28b466394b213cef6ccdaec
I20260620 23:03:56.694832 21735 ts_tablet_manager.cc:1939] T 5fa085c5c28b466394b213cef6ccdaec P fdf4f5dd9966480989eb091f8573a141: Deleting consensus metadata
Jun 20 23:03:56 dist-test-slave-f63b krb5kdc[21190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.1@KRBTEST.COM
I20260620 23:03:56.697439 21220 catalog_manager.cc:4841] TS fdf4f5dd9966480989eb091f8573a141 (127.2.237.3:41137): tablet 5fa085c5c28b466394b213cef6ccdaec (table test-table [id=bfb313a1f8c44a01b18e0f1ce07dc82b]) successfully deleted
I20260620 23:03:56.706727 21709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:56.671163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55218 (local address 127.2.237.1:42953)
0620 23:03:56.671347 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:56.671353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:56.681578 (+ 10225us) server_negotiation.cc:407] Connection header received
0620 23:03:56.682066 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:56.682073 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:56.682167 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:56.682241 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:56.684066 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.685166 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.687363 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.688036 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.688141 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:56.697703 (+ 9562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:56.697722 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:56.697743 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:56.697803 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:56.702154 (+ 4351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:56.703197 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:56.703207 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:56.703211 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:56.703422 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:56.703922 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:56.703928 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:56.703932 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:56.704430 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:56.704754 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:56.706011 (+ 1257us) server_negotiation.cc:299] Negotiation successful
0620 23:03:56.706258 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 23:03:56.707453 21419 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:03:56.827812 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 21296
I20260620 23:03:56.853745 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 21430
Jun 20 23:03:56 dist-test-slave-f63b krb5kdc[21190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.2@KRBTEST.COM
I20260620 23:03:56.872160 21707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:56.854590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57092 (local address 127.2.237.2:44773)
0620 23:03:56.854797 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:56.854803 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:56.854976 (+ 173us) server_negotiation.cc:407] Connection header received
0620 23:03:56.855159 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:56.855165 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:56.855264 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:56.855375 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:56.857172 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.858343 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.860410 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.860789 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.860884 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:56.866279 (+ 5395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:56.866291 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:56.866304 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:56.866352 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:56.869373 (+ 3021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:56.870101 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:56.870108 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:56.870111 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:56.870233 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:56.870641 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:56.870645 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:56.870647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:56.871001 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:56.871300 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:56.871607 (+ 307us) server_negotiation.cc:299] Negotiation successful
0620 23:03:56.871827 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 23:03:56.872786 21553 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:03:56.963855 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 21430
I20260620 23:03:56.981520 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 21564
Jun 20 23:03:56 dist-test-slave-f63b krb5kdc[21190](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.3@KRBTEST.COM
I20260620 23:03:56.997901 21708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:56.982391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35994 (local address 127.2.237.3:41137)
0620 23:03:56.982601 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:56.982607 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:56.982626 (+ 19us) server_negotiation.cc:407] Connection header received
0620 23:03:56.982784 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:56.982789 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:56.982872 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:56.982991 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:56.984333 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.985179 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.986661 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:56.987234 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:56.987380 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:56.991868 (+ 4488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:56.991880 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:56.991895 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:03:56.991935 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:56.995058 (+ 3123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:56.995763 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:56.995770 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:56.995772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:56.995924 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:03:56.996345 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:03:56.996351 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:03:56.996354 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:03:56.996724 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:56.997059 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:56.997315 (+ 256us) server_negotiation.cc:299] Negotiation successful
0620 23:03:56.997532 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 23:03:56.998649 21687 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:03:57.125797 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 21564
I20260620 23:03:57.145604 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 21203
I20260620 23:03:57.146688 21265 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:03:57.258994 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 21203
2026-06-20T23:03:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4382 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-20T23:03:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:03:57Z Disabled control of system clock
I20260620 23:03:57.335961 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:34421
--webserver_interface=127.2.237.62
--webserver_port=0
--builtin_ntp_servers=127.2.237.20:34993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:34421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 23:03:57.664740 21744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:57.665017 21744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:57.665096 21744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:57.672741 21744 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:03:57.672842 21744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:57.672879 21744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:57.672940 21744 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:03:57.672977 21744 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:03:57.682164 21744 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:34993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:34421
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:34421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:03:57.683481 21744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:57.685199 21744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:57.697160 21752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:57.697269 21749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:57.697291 21750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:57.698179 21744 server_base.cc:1034] running on GCE node
I20260620 23:03:57.699066 21744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:57.700573 21744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:57.701799 21744 hybrid_clock.cc:648] HybridClock initialized: now 1781996637701749 us; error 80 us; skew 500 ppm
I20260620 23:03:57.702155 21744 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 23:03:57.709197 21744 webserver.cc:460] Webserver started at http://127.2.237.62:39499/ using document root <none> and password file <none>
I20260620 23:03:57.709913 21744 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:57.710049 21744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:57.710366 21744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:57.713304 21744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "0895b03ff17948b8814016936203d966"
format_stamp: "Formatted at 2026-06-20 23:03:57 on dist-test-slave-f63b"
server_key: "951100be4ac29d37fb451b7b3d361c90"
server_key_iv: "dd717e7d88c8cd6cc24d94456215dced"
server_key_version: "encryptionkey@0"
I20260620 23:03:57.714128 21744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "0895b03ff17948b8814016936203d966"
format_stamp: "Formatted at 2026-06-20 23:03:57 on dist-test-slave-f63b"
server_key: "951100be4ac29d37fb451b7b3d361c90"
server_key_iv: "dd717e7d88c8cd6cc24d94456215dced"
server_key_version: "encryptionkey@0"
I20260620 23:03:57.719177 21744 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260620 23:03:57.722688 21758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:57.724077 21744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 23:03:57.724370 21744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "0895b03ff17948b8814016936203d966"
format_stamp: "Formatted at 2026-06-20 23:03:57 on dist-test-slave-f63b"
server_key: "951100be4ac29d37fb451b7b3d361c90"
server_key_iv: "dd717e7d88c8cd6cc24d94456215dced"
server_key_version: "encryptionkey@0"
I20260620 23:03:57.724596 21744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:57.742671 21744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:57.743566 21744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:57.743868 21744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:57.762702 21744 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:34421
I20260620 23:03:57.762792 21809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:34421 every 8 connection(s)
I20260620 23:03:57.764127 21744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:03:57.768183 21810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:57.769510 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 21744
I20260620 23:03:57.769694 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:03:57.770263 2996 external_mini_cluster.cc:1428] Setting key bf3b2a9460e8b71dd16f3151171c36ba
I20260620 23:03:57.781317 21810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0895b03ff17948b8814016936203d966: Bootstrap starting.
I20260620 23:03:57.783577 21813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:57.772258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56892 (local address 127.2.237.62:34421)
0620 23:03:57.773266 (+ 1008us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:57.773278 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:57.773318 (+ 40us) server_negotiation.cc:407] Connection header received
0620 23:03:57.775001 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:57.775022 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:57.775561 (+ 539us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:57.775929 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:57.776463 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:57.777696 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:57.779184 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:57.779848 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:57.780046 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:57.781203 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:57.781211 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:57.781224 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:57.781824 (+ 600us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:57.781900 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:57.781926 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:57.782049 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:57.782263 (+ 214us) server_negotiation.cc:299] Negotiation successful
0620 23:03:57.782562 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":321,"threads_started":1}
I20260620 23:03:57.784725 21810 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0895b03ff17948b8814016936203d966: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:57.785734 21810 log.cc:824] T 00000000000000000000000000000000 P 0895b03ff17948b8814016936203d966: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:57.788465 21810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0895b03ff17948b8814016936203d966: No bootstrap required, opened a new log
I20260620 23:03:57.795933 21810 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0895b03ff17948b8814016936203d966 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0895b03ff17948b8814016936203d966" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 34421 } }
I20260620 23:03:57.796362 21810 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0895b03ff17948b8814016936203d966 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:57.796482 21810 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0895b03ff17948b8814016936203d966 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0895b03ff17948b8814016936203d966, State: Initialized, Role: FOLLOWER
I20260620 23:03:57.797194 21810 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0895b03ff17948b8814016936203d966 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0895b03ff17948b8814016936203d966" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 34421 } }
I20260620 23:03:57.797462 21810 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0895b03ff17948b8814016936203d966 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:03:57.797636 21810 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0895b03ff17948b8814016936203d966 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:03:57.797821 21810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0895b03ff17948b8814016936203d966 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:57.800041 21810 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0895b03ff17948b8814016936203d966 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0895b03ff17948b8814016936203d966" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 34421 } }
I20260620 23:03:57.800544 21810 leader_election.cc:302] T 00000000000000000000000000000000 P 0895b03ff17948b8814016936203d966 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0895b03ff17948b8814016936203d966; no voters:
I20260620 23:03:57.801159 21810 leader_election.cc:288] T 00000000000000000000000000000000 P 0895b03ff17948b8814016936203d966 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:03:57.801252 21815 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0895b03ff17948b8814016936203d966 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:57.802129 21815 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0895b03ff17948b8814016936203d966 [term 1 LEADER]: Becoming Leader. State: Replica: 0895b03ff17948b8814016936203d966, State: Running, Role: LEADER
I20260620 23:03:57.802661 21815 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0895b03ff17948b8814016936203d966 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0895b03ff17948b8814016936203d966" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 34421 } }
I20260620 23:03:57.803347 21810 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0895b03ff17948b8814016936203d966 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:03:57.805718 21817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0895b03ff17948b8814016936203d966 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0895b03ff17948b8814016936203d966. Latest consensus state: current_term: 1 leader_uuid: "0895b03ff17948b8814016936203d966" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0895b03ff17948b8814016936203d966" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 34421 } } }
I20260620 23:03:57.805788 21816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0895b03ff17948b8814016936203d966 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0895b03ff17948b8814016936203d966" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0895b03ff17948b8814016936203d966" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 34421 } } }
I20260620 23:03:57.806116 21817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0895b03ff17948b8814016936203d966 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:57.806121 21816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0895b03ff17948b8814016936203d966 [sys.catalog]: This master's current role is: LEADER
I20260620 23:03:57.807977 21823 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:03:57.814900 21823 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:03:57.824958 21823 catalog_manager.cc:1347] Generated new cluster ID: da30978271104a67a6faffa61a459e1a
I20260620 23:03:57.825069 21823 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:03:57.852062 21823 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:03:57.853408 21823 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:03:57.875330 21823 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0895b03ff17948b8814016936203d966: Generated new TSK 0
I20260620 23:03:57.876317 21823 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:03:57.887516 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:0
--local_ip_for_outbound_sockets=127.2.237.1
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:34421
--builtin_ntp_servers=127.2.237.20:34993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 23:03:58.222874 21834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:58.223151 21834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:58.223239 21834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:58.230726 21834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:58.230820 21834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:58.230926 21834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 23:03:58.239995 21834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:34993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:34421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:03:58.241285 21834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:58.243148 21834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:58.256301 21840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:58.256376 21839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:58.256515 21842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:58.256916 21834 server_base.cc:1034] running on GCE node
I20260620 23:03:58.257717 21834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:58.258708 21834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:58.259925 21834 hybrid_clock.cc:648] HybridClock initialized: now 1781996638259897 us; error 48 us; skew 500 ppm
I20260620 23:03:58.260272 21834 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 23:03:58.262802 21834 webserver.cc:460] Webserver started at http://127.2.237.1:43825/ using document root <none> and password file <none>
I20260620 23:03:58.263505 21834 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:58.263593 21834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:58.263939 21834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:58.266873 21834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-0/data/instance:
uuid: "890631203877422d8e4542789f125191"
format_stamp: "Formatted at 2026-06-20 23:03:58 on dist-test-slave-f63b"
server_key: "95b21f1239c69926393e6b8f8808709d"
server_key_iv: "9c45728f936b2b371630b5e21ed2e7fc"
server_key_version: "encryptionkey@0"
I20260620 23:03:58.267686 21834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance:
uuid: "890631203877422d8e4542789f125191"
format_stamp: "Formatted at 2026-06-20 23:03:58 on dist-test-slave-f63b"
server_key: "95b21f1239c69926393e6b8f8808709d"
server_key_iv: "9c45728f936b2b371630b5e21ed2e7fc"
server_key_version: "encryptionkey@0"
I20260620 23:03:58.272843 21834 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260620 23:03:58.276397 21848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:58.277740 21834 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 23:03:58.277915 21834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "890631203877422d8e4542789f125191"
format_stamp: "Formatted at 2026-06-20 23:03:58 on dist-test-slave-f63b"
server_key: "95b21f1239c69926393e6b8f8808709d"
server_key_iv: "9c45728f936b2b371630b5e21ed2e7fc"
server_key_version: "encryptionkey@0"
I20260620 23:03:58.278131 21834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:58.302891 21834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:58.303825 21834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:58.304127 21834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:58.305178 21834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:58.306740 21834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:58.306854 21834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:58.306980 21834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:58.307082 21834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:58.338806 21834 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:46049
I20260620 23:03:58.338905 21960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:46049 every 8 connection(s)
I20260620 23:03:58.340085 21834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 23:03:58.342147 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 21834
I20260620 23:03:58.342324 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance
I20260620 23:03:58.343044 2996 external_mini_cluster.cc:1428] Setting key bf98353813ecb30c131441a5a2225ab7
I20260620 23:03:58.346074 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:0
--local_ip_for_outbound_sockets=127.2.237.2
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:34421
--builtin_ntp_servers=127.2.237.20:34993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 23:03:58.353737 21963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:58.343111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:57881 (local address 127.2.237.62:34421)
0620 23:03:58.343669 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:58.343676 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:58.345136 (+ 1460us) server_negotiation.cc:407] Connection header received
0620 23:03:58.347191 (+ 2055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:58.347196 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:58.347290 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:58.347426 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:58.348526 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:58.349246 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:58.350986 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:58.351378 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:58.351517 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:58.352942 (+ 1425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:58.352949 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:58.352958 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:58.352981 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:58.353035 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:58.353053 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:58.353150 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:58.353365 (+ 215us) server_negotiation.cc:299] Negotiation successful
0620 23:03:58.353489 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":300,"threads_started":1}
I20260620 23:03:58.354985 21964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:58.343756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:34421 (local address 127.2.237.1:57881)
0620 23:03:58.344942 (+ 1186us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:58.345017 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:58.346920 (+ 1903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:58.347677 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:58.347686 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:58.347784 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:58.348356 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:58.348372 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:58.349397 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:58.349407 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:58.350839 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:58.350852 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:58.351621 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:58.351628 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:58.351857 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:58.352616 (+ 759us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:03:58.352619 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:58.352658 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:03:58.352736 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:58.353155 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:58.353164 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:58.353170 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 23:03:58.353311 (+ 141us) client_negotiation.cc:240] Negotiation successful
0620 23:03:58.353605 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":355,"threads_started":1}
I20260620 23:03:58.356554 21961 heartbeater.cc:347] Connected to a master server at 127.2.237.62:34421
I20260620 23:03:58.356854 21961 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:58.357604 21961 heartbeater.cc:511] Master 127.2.237.62:34421 requested a full tablet report, sending...
I20260620 23:03:58.359589 21775 ts_manager.cc:194] Registered new tserver with Master: 890631203877422d8e4542789f125191 (127.2.237.1:46049)
I20260620 23:03:58.361034 21775 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.237.1:57881
W20260620 23:03:58.669866 21965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:58.670149 21965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:58.670320 21965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:58.677577 21965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:58.677676 21965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:58.677845 21965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 23:03:58.687199 21965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:34993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:34421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:03:58.688553 21965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:58.690325 21965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:58.703876 21971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:58.703959 21970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:58.705559 21973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:58.706269 21965 server_base.cc:1034] running on GCE node
I20260620 23:03:58.707052 21965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:58.708241 21965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:58.709497 21965 hybrid_clock.cc:648] HybridClock initialized: now 1781996638709392 us; error 90 us; skew 500 ppm
I20260620 23:03:58.709932 21965 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 23:03:58.712630 21965 webserver.cc:460] Webserver started at http://127.2.237.2:44239/ using document root <none> and password file <none>
I20260620 23:03:58.713289 21965 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:58.713382 21965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:58.713639 21965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:58.716404 21965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-1/data/instance:
uuid: "f8d82b9d91a144a0aea2519a88b89c44"
format_stamp: "Formatted at 2026-06-20 23:03:58 on dist-test-slave-f63b"
server_key: "ee0170d6ed86ca058e50c427c8819b1f"
server_key_iv: "608492d225e5f90b86a0424b4318a6d1"
server_key_version: "encryptionkey@0"
I20260620 23:03:58.717110 21965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance:
uuid: "f8d82b9d91a144a0aea2519a88b89c44"
format_stamp: "Formatted at 2026-06-20 23:03:58 on dist-test-slave-f63b"
server_key: "ee0170d6ed86ca058e50c427c8819b1f"
server_key_iv: "608492d225e5f90b86a0424b4318a6d1"
server_key_version: "encryptionkey@0"
I20260620 23:03:58.722029 21965 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260620 23:03:58.725636 21979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:58.727102 21965 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 23:03:58.727274 21965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "f8d82b9d91a144a0aea2519a88b89c44"
format_stamp: "Formatted at 2026-06-20 23:03:58 on dist-test-slave-f63b"
server_key: "ee0170d6ed86ca058e50c427c8819b1f"
server_key_iv: "608492d225e5f90b86a0424b4318a6d1"
server_key_version: "encryptionkey@0"
I20260620 23:03:58.727506 21965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:58.745141 21965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:58.745903 21965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:58.746155 21965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:58.747134 21965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:58.748530 21965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:58.748615 21965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:58.748688 21965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:58.748744 21965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:58.784564 21965 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:40771
I20260620 23:03:58.784653 22091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:40771 every 8 connection(s)
I20260620 23:03:58.785898 21965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 23:03:58.791291 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 21965
I20260620 23:03:58.791488 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance
I20260620 23:03:58.792057 2996 external_mini_cluster.cc:1428] Setting key c42b5afcc7ace02fa47aee0de2abb135
I20260620 23:03:58.795039 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:0
--local_ip_for_outbound_sockets=127.2.237.3
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:34421
--builtin_ntp_servers=127.2.237.20:34993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 23:03:58.798923 21963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:58.788435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:35843 (local address 127.2.237.62:34421)
0620 23:03:58.788606 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:58.788610 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:58.789886 (+ 1276us) server_negotiation.cc:407] Connection header received
0620 23:03:58.791755 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:58.791762 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:58.791850 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:58.791964 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:58.793113 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:58.794085 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:58.795732 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:58.796547 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:58.796689 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:58.797998 (+ 1309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:58.798008 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:58.798015 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:58.798040 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:58.798084 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:58.798103 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:58.798193 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:58.798400 (+ 207us) server_negotiation.cc:299] Negotiation successful
0620 23:03:58.798507 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260620 23:03:58.799779 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:58.788893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:34421 (local address 127.2.237.2:35843)
0620 23:03:58.789745 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:58.789778 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:58.791469 (+ 1691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:58.792177 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:58.792187 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:58.792271 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:58.792988 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:58.793010 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:58.794224 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:58.794228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:58.795591 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:58.795603 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:58.796755 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:58.796760 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:58.796957 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:58.797671 (+ 714us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:03:58.797674 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:58.797713 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:03:58.797784 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:58.798199 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:58.798206 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:58.798212 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 23:03:58.798350 (+ 138us) client_negotiation.cc:240] Negotiation successful
0620 23:03:58.798699 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":260,"threads_started":1}
I20260620 23:03:58.801591 22092 heartbeater.cc:347] Connected to a master server at 127.2.237.62:34421
I20260620 23:03:58.801882 22092 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:58.802664 22092 heartbeater.cc:511] Master 127.2.237.62:34421 requested a full tablet report, sending...
I20260620 23:03:58.804334 21775 ts_manager.cc:194] Registered new tserver with Master: f8d82b9d91a144a0aea2519a88b89c44 (127.2.237.2:40771)
I20260620 23:03:58.805214 21775 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.237.2:35843
W20260620 23:03:59.130057 22095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:03:59.130331 22095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:03:59.130511 22095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:03:59.137878 22095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:03:59.137990 22095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:03:59.138190 22095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 23:03:59.147274 22095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:34993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:34421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:03:59.148649 22095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:03:59.150450 22095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:03:59.163866 22100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:59.163893 22103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:03:59.163939 22101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:03:59.165071 22095 server_base.cc:1034] running on GCE node
I20260620 23:03:59.165704 22095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:03:59.166805 22095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:03:59.168062 22095 hybrid_clock.cc:648] HybridClock initialized: now 1781996639168008 us; error 45 us; skew 500 ppm
I20260620 23:03:59.168428 22095 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 23:03:59.170897 22095 webserver.cc:460] Webserver started at http://127.2.237.3:33715/ using document root <none> and password file <none>
I20260620 23:03:59.171643 22095 fs_manager.cc:359] Metadata directory not provided
I20260620 23:03:59.171780 22095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:03:59.172114 22095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:03:59.175149 22095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-2/data/instance:
uuid: "a654fc35397c4506a84d487f2a1d7655"
format_stamp: "Formatted at 2026-06-20 23:03:59 on dist-test-slave-f63b"
server_key: "c4319ad7189feb787bbbb2dbfdcb46f4"
server_key_iv: "bf566e0304539eac950be3ddbb69eba1"
server_key_version: "encryptionkey@0"
I20260620 23:03:59.175964 22095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance:
uuid: "a654fc35397c4506a84d487f2a1d7655"
format_stamp: "Formatted at 2026-06-20 23:03:59 on dist-test-slave-f63b"
server_key: "c4319ad7189feb787bbbb2dbfdcb46f4"
server_key_iv: "bf566e0304539eac950be3ddbb69eba1"
server_key_version: "encryptionkey@0"
I20260620 23:03:59.181305 22095 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260620 23:03:59.185370 22109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:59.186945 22095 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 23:03:59.187130 22095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "a654fc35397c4506a84d487f2a1d7655"
format_stamp: "Formatted at 2026-06-20 23:03:59 on dist-test-slave-f63b"
server_key: "c4319ad7189feb787bbbb2dbfdcb46f4"
server_key_iv: "bf566e0304539eac950be3ddbb69eba1"
server_key_version: "encryptionkey@0"
I20260620 23:03:59.187361 22095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:03:59.210853 22095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:03:59.211830 22095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:03:59.212114 22095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:03:59.213215 22095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:03:59.214771 22095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:03:59.214933 22095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:59.215063 22095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:03:59.215152 22095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:03:59.249171 22095 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:41293
I20260620 23:03:59.249233 22221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:41293 every 8 connection(s)
I20260620 23:03:59.250483 22095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 23:03:59.252462 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 22095
I20260620 23:03:59.252643 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance
I20260620 23:03:59.253234 2996 external_mini_cluster.cc:1428] Setting key ee1bb0fd32b5c152519198f1d7e16cde
I20260620 23:03:59.263685 21963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:59.253374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:38775 (local address 127.2.237.62:34421)
0620 23:03:59.253588 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:59.253594 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:59.255224 (+ 1630us) server_negotiation.cc:407] Connection header received
0620 23:03:59.257047 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:59.257051 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:59.257118 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:59.257220 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:59.258461 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.259496 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.261106 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.261526 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.261623 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:59.263019 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:59.263024 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:59.263027 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:59.263044 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:59.263074 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:59.263086 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:59.263154 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:59.263362 (+ 208us) server_negotiation.cc:299] Negotiation successful
0620 23:03:59.263440 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260620 23:03:59.264714 22224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:59.253902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:34421 (local address 127.2.237.3:38775)
0620 23:03:59.255075 (+ 1173us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:59.255122 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:59.256784 (+ 1662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:59.257495 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:59.257505 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:59.257609 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:03:59.258258 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:59.258277 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.259642 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.259646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:59.260973 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:59.260985 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.261711 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.261716 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:59.261911 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:59.262707 (+ 796us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:03:59.262710 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:03:59.262756 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:03:59.262841 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:03:59.263169 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:03:59.263177 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:03:59.263183 (+ 6us) client_negotiation.cc:769] Sending connection context
0620 23:03:59.263328 (+ 145us) client_negotiation.cc:240] Negotiation successful
0620 23:03:59.263634 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":352,"threads_started":1}
I20260620 23:03:59.266211 22222 heartbeater.cc:347] Connected to a master server at 127.2.237.62:34421
I20260620 23:03:59.266535 22222 heartbeater.cc:464] Registering TS with master...
I20260620 23:03:59.267426 22222 heartbeater.cc:511] Master 127.2.237.62:34421 requested a full tablet report, sending...
I20260620 23:03:59.268803 21775 ts_manager.cc:194] Registered new tserver with Master: a654fc35397c4506a84d487f2a1d7655 (127.2.237.3:41293)
I20260620 23:03:59.269564 21775 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.237.3:38775
I20260620 23:03:59.281162 2996 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:03:59.292026 21963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:59.285629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56900 (local address 127.2.237.62:34421)
0620 23:03:59.285960 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:59.285969 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:59.286233 (+ 264us) server_negotiation.cc:407] Connection header received
0620 23:03:59.286480 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:59.286484 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:59.286562 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:59.286707 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:59.287271 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.288090 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.289402 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.289804 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.289942 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:59.291237 (+ 1295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:59.291243 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:59.291246 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:59.291266 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:59.291299 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:59.291320 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:59.291486 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:59.291726 (+ 240us) server_negotiation.cc:299] Negotiation successful
0620 23:03:59.291801 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260620 23:03:59.296764 21775 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56900:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:03:59.304179 21775 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:03:59.328168 22235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:59.317386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40900 (local address 127.2.237.3:41293)
0620 23:03:59.320068 (+ 2682us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:59.320080 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:59.320124 (+ 44us) server_negotiation.cc:407] Connection header received
0620 23:03:59.320241 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:59.320249 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:59.320564 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:59.320790 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:59.321675 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.323250 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.325556 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.326992 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.327145 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:59.327445 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:59.327671 (+ 226us) server_negotiation.cc:299] Negotiation successful
0620 23:03:59.327836 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2459,"thread_start_us":2299,"threads_started":1}
I20260620 23:03:59.328210 22234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:59.317053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:41293 (local address 127.0.0.1:40900)
0620 23:03:59.318026 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:59.318082 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:59.318342 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:59.320835 (+ 2493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:59.320846 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:59.320899 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:59.321421 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:59.321437 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.323378 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.323386 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:59.325309 (+ 1923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:59.325327 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.327187 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.327193 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:59.327381 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:59.327441 (+ 60us) client_negotiation.cc:769] Sending connection context
0620 23:03:59.327630 (+ 189us) client_negotiation.cc:240] Negotiation successful
0620 23:03:59.327816 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":321,"threads_started":1}
I20260620 23:03:59.330438 22237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:59.318175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:46049 (local address 127.0.0.1:57372)
0620 23:03:59.321565 (+ 3390us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:59.321591 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:59.321720 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:59.322507 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:59.322512 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:59.322536 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:59.322886 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:59.322898 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.324857 (+ 1959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.324864 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:59.327048 (+ 2184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:59.327063 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.329510 (+ 2447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.329518 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:59.329737 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:59.329777 (+ 40us) client_negotiation.cc:769] Sending connection context
0620 23:03:59.329869 (+ 92us) client_negotiation.cc:240] Negotiation successful
0620 23:03:59.330054 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3245,"mutex_wait_us":42,"thread_start_us":3072,"threads_started":1}
I20260620 23:03:59.330639 22239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:59.318744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57372 (local address 127.2.237.1:46049)
0620 23:03:59.321338 (+ 2594us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:59.321348 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:59.321889 (+ 541us) server_negotiation.cc:407] Connection header received
0620 23:03:59.322025 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:59.322033 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:59.322318 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:59.322518 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:59.323071 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.324659 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.327300 (+ 2641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.328936 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.329129 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:59.329346 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:59.329970 (+ 624us) server_negotiation.cc:299] Negotiation successful
0620 23:03:59.330139 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2391,"thread_start_us":2275,"threads_started":1}
I20260620 23:03:59.330662 22238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:59.318228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42310 (local address 127.2.237.2:40771)
0620 23:03:59.320346 (+ 2118us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:59.320354 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:59.320380 (+ 26us) server_negotiation.cc:407] Connection header received
0620 23:03:59.320501 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:59.320511 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:59.320736 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:59.322261 (+ 1525us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:59.322543 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.324207 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.326849 (+ 2642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.328924 (+ 2075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.329080 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:59.329342 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:59.330082 (+ 740us) server_negotiation.cc:299] Negotiation successful
0620 23:03:59.330258 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1966,"thread_start_us":1859,"threads_started":1}
I20260620 23:03:59.331329 22236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:59.317775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:40771 (local address 127.0.0.1:42310)
0620 23:03:59.319217 (+ 1442us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:59.319242 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:59.319389 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:59.321486 (+ 2097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:59.321492 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:59.321519 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:59.322043 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:59.322057 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.324359 (+ 2302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.324365 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:59.326638 (+ 2273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:59.326652 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.329524 (+ 2872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.329529 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:59.329820 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:59.329865 (+ 45us) client_negotiation.cc:769] Sending connection context
0620 23:03:59.329960 (+ 95us) client_negotiation.cc:240] Negotiation successful
0620 23:03:59.330993 (+ 1033us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1273,"mutex_wait_us":877,"thread_start_us":1077,"threads_started":1}
I20260620 23:03:59.333004 22157 tablet_service.cc:1467] Processing CreateTablet for tablet 016b2b438b134f2f9d2ade5da8785dec (DEFAULT_TABLE table=test-table [id=72017230891a4f91917146b836fddaa5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:59.334556 22157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 016b2b438b134f2f9d2ade5da8785dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:59.334559 21896 tablet_service.cc:1467] Processing CreateTablet for tablet 016b2b438b134f2f9d2ade5da8785dec (DEFAULT_TABLE table=test-table [id=72017230891a4f91917146b836fddaa5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:59.335028 22027 tablet_service.cc:1467] Processing CreateTablet for tablet 016b2b438b134f2f9d2ade5da8785dec (DEFAULT_TABLE table=test-table [id=72017230891a4f91917146b836fddaa5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:03:59.335976 21896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 016b2b438b134f2f9d2ade5da8785dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:59.336306 22027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 016b2b438b134f2f9d2ade5da8785dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:03:59.351230 22240 tablet_bootstrap.cc:492] T 016b2b438b134f2f9d2ade5da8785dec P 890631203877422d8e4542789f125191: Bootstrap starting.
I20260620 23:03:59.354830 22240 tablet_bootstrap.cc:654] T 016b2b438b134f2f9d2ade5da8785dec P 890631203877422d8e4542789f125191: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:59.355265 22242 tablet_bootstrap.cc:492] T 016b2b438b134f2f9d2ade5da8785dec P a654fc35397c4506a84d487f2a1d7655: Bootstrap starting.
I20260620 23:03:59.356720 22240 log.cc:824] T 016b2b438b134f2f9d2ade5da8785dec P 890631203877422d8e4542789f125191: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:59.356809 22241 tablet_bootstrap.cc:492] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44: Bootstrap starting.
I20260620 23:03:59.359269 22242 tablet_bootstrap.cc:654] T 016b2b438b134f2f9d2ade5da8785dec P a654fc35397c4506a84d487f2a1d7655: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:59.360294 22241 tablet_bootstrap.cc:654] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44: Neither blocks nor log segments found. Creating new log.
I20260620 23:03:59.360754 22242 log.cc:824] T 016b2b438b134f2f9d2ade5da8785dec P a654fc35397c4506a84d487f2a1d7655: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:59.361718 22241 log.cc:824] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44: Log is configured to *not* fsync() on all Append() calls
I20260620 23:03:59.363334 22240 tablet_bootstrap.cc:492] T 016b2b438b134f2f9d2ade5da8785dec P 890631203877422d8e4542789f125191: No bootstrap required, opened a new log
I20260620 23:03:59.363673 22240 ts_tablet_manager.cc:1397] T 016b2b438b134f2f9d2ade5da8785dec P 890631203877422d8e4542789f125191: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.004s
I20260620 23:03:59.363888 21961 heartbeater.cc:503] Master 127.2.237.62:34421 was elected leader, sending a full tablet report...
I20260620 23:03:59.364849 22241 tablet_bootstrap.cc:492] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44: No bootstrap required, opened a new log
I20260620 23:03:59.364916 22242 tablet_bootstrap.cc:492] T 016b2b438b134f2f9d2ade5da8785dec P a654fc35397c4506a84d487f2a1d7655: No bootstrap required, opened a new log
I20260620 23:03:59.365113 22241 ts_tablet_manager.cc:1397] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260620 23:03:59.365191 22242 ts_tablet_manager.cc:1397] T 016b2b438b134f2f9d2ade5da8785dec P a654fc35397c4506a84d487f2a1d7655: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260620 23:03:59.374778 22242 raft_consensus.cc:348] T 016b2b438b134f2f9d2ade5da8785dec P a654fc35397c4506a84d487f2a1d7655 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a654fc35397c4506a84d487f2a1d7655" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 41293 } } peers { permanent_uuid: "890631203877422d8e4542789f125191" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 46049 } } peers { permanent_uuid: "f8d82b9d91a144a0aea2519a88b89c44" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 40771 } }
I20260620 23:03:59.374783 22240 raft_consensus.cc:348] T 016b2b438b134f2f9d2ade5da8785dec P 890631203877422d8e4542789f125191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a654fc35397c4506a84d487f2a1d7655" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 41293 } } peers { permanent_uuid: "890631203877422d8e4542789f125191" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 46049 } } peers { permanent_uuid: "f8d82b9d91a144a0aea2519a88b89c44" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 40771 } }
I20260620 23:03:59.375236 22242 raft_consensus.cc:374] T 016b2b438b134f2f9d2ade5da8785dec P a654fc35397c4506a84d487f2a1d7655 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:59.375236 22240 raft_consensus.cc:374] T 016b2b438b134f2f9d2ade5da8785dec P 890631203877422d8e4542789f125191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:59.375334 22242 raft_consensus.cc:729] T 016b2b438b134f2f9d2ade5da8785dec P a654fc35397c4506a84d487f2a1d7655 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a654fc35397c4506a84d487f2a1d7655, State: Initialized, Role: FOLLOWER
I20260620 23:03:59.375332 22240 raft_consensus.cc:729] T 016b2b438b134f2f9d2ade5da8785dec P 890631203877422d8e4542789f125191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 890631203877422d8e4542789f125191, State: Initialized, Role: FOLLOWER
I20260620 23:03:59.375725 22241 raft_consensus.cc:348] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a654fc35397c4506a84d487f2a1d7655" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 41293 } } peers { permanent_uuid: "890631203877422d8e4542789f125191" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 46049 } } peers { permanent_uuid: "f8d82b9d91a144a0aea2519a88b89c44" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 40771 } }
I20260620 23:03:59.376139 22241 raft_consensus.cc:374] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:03:59.376237 22241 raft_consensus.cc:729] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8d82b9d91a144a0aea2519a88b89c44, State: Initialized, Role: FOLLOWER
I20260620 23:03:59.376150 22242 consensus_queue.cc:260] T 016b2b438b134f2f9d2ade5da8785dec P a654fc35397c4506a84d487f2a1d7655 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a654fc35397c4506a84d487f2a1d7655" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 41293 } } peers { permanent_uuid: "890631203877422d8e4542789f125191" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 46049 } } peers { permanent_uuid: "f8d82b9d91a144a0aea2519a88b89c44" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 40771 } }
I20260620 23:03:59.376868 22241 consensus_queue.cc:260] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a654fc35397c4506a84d487f2a1d7655" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 41293 } } peers { permanent_uuid: "890631203877422d8e4542789f125191" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 46049 } } peers { permanent_uuid: "f8d82b9d91a144a0aea2519a88b89c44" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 40771 } }
I20260620 23:03:59.377573 22222 heartbeater.cc:503] Master 127.2.237.62:34421 was elected leader, sending a full tablet report...
I20260620 23:03:59.376140 22240 consensus_queue.cc:260] T 016b2b438b134f2f9d2ade5da8785dec P 890631203877422d8e4542789f125191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a654fc35397c4506a84d487f2a1d7655" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 41293 } } peers { permanent_uuid: "890631203877422d8e4542789f125191" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 46049 } } peers { permanent_uuid: "f8d82b9d91a144a0aea2519a88b89c44" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 40771 } }
I20260620 23:03:59.378291 22242 ts_tablet_manager.cc:1428] T 016b2b438b134f2f9d2ade5da8785dec P a654fc35397c4506a84d487f2a1d7655: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
I20260620 23:03:59.380100 22092 heartbeater.cc:503] Master 127.2.237.62:34421 was elected leader, sending a full tablet report...
I20260620 23:03:59.380942 22241 ts_tablet_manager.cc:1428] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260620 23:03:59.382812 22240 ts_tablet_manager.cc:1428] T 016b2b438b134f2f9d2ade5da8785dec P 890631203877422d8e4542789f125191: Time spent starting tablet: real 0.019s user 0.008s sys 0.007s
W20260620 23:03:59.502188 22223 tablet.cc:2369] T 016b2b438b134f2f9d2ade5da8785dec P a654fc35397c4506a84d487f2a1d7655: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:03:59.505218 22248 raft_consensus.cc:482] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:03:59.505528 22248 raft_consensus.cc:504] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a654fc35397c4506a84d487f2a1d7655" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 41293 } } peers { permanent_uuid: "890631203877422d8e4542789f125191" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 46049 } } peers { permanent_uuid: "f8d82b9d91a144a0aea2519a88b89c44" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 40771 } }
I20260620 23:03:59.507336 22248 leader_election.cc:288] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a654fc35397c4506a84d487f2a1d7655 (127.2.237.3:41293), 890631203877422d8e4542789f125191 (127.2.237.1:46049)
I20260620 23:03:59.514699 22249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:59.507752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:41293 (local address 127.2.237.2:37163)
0620 23:03:59.508523 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:59.508546 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:59.508759 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:59.509215 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:59.509218 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:59.509243 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:59.509635 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:59.509649 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.510992 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.510998 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:59.512733 (+ 1735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:59.512745 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.513888 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.513894 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:59.514069 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:59.514092 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 23:03:59.514194 (+ 102us) client_negotiation.cc:240] Negotiation successful
0620 23:03:59.514306 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"mutex_wait_us":63,"thread_start_us":431,"threads_started":1}
I20260620 23:03:59.515151 22239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:59.508784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:39749 (local address 127.2.237.1:46049)
0620 23:03:59.509043 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:59.509049 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:59.509439 (+ 390us) server_negotiation.cc:407] Connection header received
0620 23:03:59.509570 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:59.509576 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:59.509669 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:59.509761 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:59.510374 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.511258 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.513230 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.514143 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.514305 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:59.514399 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:59.514763 (+ 364us) server_negotiation.cc:299] Negotiation successful
0620 23:03:59.514907 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260620 23:03:59.515309 22235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:59.507882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:37163 (local address 127.2.237.3:41293)
0620 23:03:59.508149 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:59.508156 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:59.508643 (+ 487us) server_negotiation.cc:407] Connection header received
0620 23:03:59.508915 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:59.508923 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:59.509018 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:59.509118 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:03:59.509823 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.510864 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.512925 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.513702 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.513851 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:59.513988 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:59.514694 (+ 706us) server_negotiation.cc:299] Negotiation successful
0620 23:03:59.515078 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260620 23:03:59.515882 22250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:59.508537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:46049 (local address 127.2.237.2:39749)
0620 23:03:59.509335 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:59.509349 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 23:03:59.509450 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:03:59.509823 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:03:59.509828 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:03:59.509854 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:03:59.510227 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:59.510240 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.511500 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.511507 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:59.513043 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:03:59.513055 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.514349 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.514355 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:03:59.514525 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:59.514548 (+ 23us) client_negotiation.cc:769] Sending connection context
0620 23:03:59.515330 (+ 782us) client_negotiation.cc:240] Negotiation successful
0620 23:03:59.515478 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":493,"threads_started":1}
I20260620 23:03:59.516283 22177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "016b2b438b134f2f9d2ade5da8785dec" candidate_uuid: "f8d82b9d91a144a0aea2519a88b89c44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a654fc35397c4506a84d487f2a1d7655" is_pre_election: true
I20260620 23:03:59.516779 22177 raft_consensus.cc:2454] T 016b2b438b134f2f9d2ade5da8785dec P a654fc35397c4506a84d487f2a1d7655 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8d82b9d91a144a0aea2519a88b89c44 in term 0.
I20260620 23:03:59.516794 21916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "016b2b438b134f2f9d2ade5da8785dec" candidate_uuid: "f8d82b9d91a144a0aea2519a88b89c44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "890631203877422d8e4542789f125191" is_pre_election: true
I20260620 23:03:59.517323 21916 raft_consensus.cc:2454] T 016b2b438b134f2f9d2ade5da8785dec P 890631203877422d8e4542789f125191 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8d82b9d91a144a0aea2519a88b89c44 in term 0.
I20260620 23:03:59.517458 21983 leader_election.cc:302] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a654fc35397c4506a84d487f2a1d7655, f8d82b9d91a144a0aea2519a88b89c44; no voters:
I20260620 23:03:59.517871 22248 raft_consensus.cc:2781] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:03:59.518020 22248 raft_consensus.cc:482] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:03:59.518129 22248 raft_consensus.cc:3037] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:59.520334 22248 raft_consensus.cc:504] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a654fc35397c4506a84d487f2a1d7655" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 41293 } } peers { permanent_uuid: "890631203877422d8e4542789f125191" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 46049 } } peers { permanent_uuid: "f8d82b9d91a144a0aea2519a88b89c44" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 40771 } }
I20260620 23:03:59.520900 22248 leader_election.cc:288] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44 [CANDIDATE]: Term 1 election: Requested vote from peers a654fc35397c4506a84d487f2a1d7655 (127.2.237.3:41293), 890631203877422d8e4542789f125191 (127.2.237.1:46049)
I20260620 23:03:59.521322 22177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "016b2b438b134f2f9d2ade5da8785dec" candidate_uuid: "f8d82b9d91a144a0aea2519a88b89c44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a654fc35397c4506a84d487f2a1d7655"
I20260620 23:03:59.521477 21916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "016b2b438b134f2f9d2ade5da8785dec" candidate_uuid: "f8d82b9d91a144a0aea2519a88b89c44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "890631203877422d8e4542789f125191"
I20260620 23:03:59.521529 22177 raft_consensus.cc:3037] T 016b2b438b134f2f9d2ade5da8785dec P a654fc35397c4506a84d487f2a1d7655 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:59.521688 21916 raft_consensus.cc:3037] T 016b2b438b134f2f9d2ade5da8785dec P 890631203877422d8e4542789f125191 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:03:59.523808 22177 raft_consensus.cc:2454] T 016b2b438b134f2f9d2ade5da8785dec P a654fc35397c4506a84d487f2a1d7655 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8d82b9d91a144a0aea2519a88b89c44 in term 1.
I20260620 23:03:59.524063 21916 raft_consensus.cc:2454] T 016b2b438b134f2f9d2ade5da8785dec P 890631203877422d8e4542789f125191 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8d82b9d91a144a0aea2519a88b89c44 in term 1.
I20260620 23:03:59.524243 21983 leader_election.cc:302] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a654fc35397c4506a84d487f2a1d7655, f8d82b9d91a144a0aea2519a88b89c44; no voters:
I20260620 23:03:59.524552 22248 raft_consensus.cc:2781] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:03:59.524997 22248 raft_consensus.cc:686] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44 [term 1 LEADER]: Becoming Leader. State: Replica: f8d82b9d91a144a0aea2519a88b89c44, State: Running, Role: LEADER
I20260620 23:03:59.525473 22248 consensus_queue.cc:237] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a654fc35397c4506a84d487f2a1d7655" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 41293 } } peers { permanent_uuid: "890631203877422d8e4542789f125191" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 46049 } } peers { permanent_uuid: "f8d82b9d91a144a0aea2519a88b89c44" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 40771 } }
I20260620 23:03:59.530385 21774 catalog_manager.cc:5495] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44 reported cstate change: term changed from 0 to 1, leader changed from <none> to f8d82b9d91a144a0aea2519a88b89c44 (127.2.237.2). New cstate: current_term: 1 leader_uuid: "f8d82b9d91a144a0aea2519a88b89c44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a654fc35397c4506a84d487f2a1d7655" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 41293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "890631203877422d8e4542789f125191" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 46049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8d82b9d91a144a0aea2519a88b89c44" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 40771 } health_report { overall_health: HEALTHY } } }
W20260620 23:03:59.537967 22093 tablet.cc:2369] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:03:59.573460 22238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:59.568206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42314 (local address 127.2.237.2:40771)
0620 23:03:59.568406 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:59.568411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:59.568429 (+ 18us) server_negotiation.cc:407] Connection header received
0620 23:03:59.568517 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:59.568520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:59.568615 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:59.568752 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:03:59.569084 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.569815 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.571323 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.571762 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.571867 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:59.572299 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:03:59.572827 (+ 528us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:03:59.572975 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:59.573110 (+ 135us) server_negotiation.cc:299] Negotiation successful
0620 23:03:59.573173 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260620 23:03:59.581440 21916 raft_consensus.cc:1261] T 016b2b438b134f2f9d2ade5da8785dec P 890631203877422d8e4542789f125191 [term 1 FOLLOWER]: Refusing update from remote peer f8d82b9d91a144a0aea2519a88b89c44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:59.581516 22177 raft_consensus.cc:1261] T 016b2b438b134f2f9d2ade5da8785dec P a654fc35397c4506a84d487f2a1d7655 [term 1 FOLLOWER]: Refusing update from remote peer f8d82b9d91a144a0aea2519a88b89c44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:03:59.582460 22251 consensus_queue.cc:1035] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "890631203877422d8e4542789f125191" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 46049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:03:59.582857 22248 consensus_queue.cc:1035] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a654fc35397c4506a84d487f2a1d7655" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 41293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 23:03:59.592717 21962 tablet.cc:2369] T 016b2b438b134f2f9d2ade5da8785dec P 890631203877422d8e4542789f125191: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:03:59.613305 21774 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:03:59.613647 21774 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56900:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:03:59.617717 21774 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0895b03ff17948b8814016936203d966: Sending DeleteTablet for 3 replicas of tablet 016b2b438b134f2f9d2ade5da8785dec
I20260620 23:03:59.619122 21896 tablet_service.cc:1514] Processing DeleteTablet for tablet 016b2b438b134f2f9d2ade5da8785dec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:59 UTC) from {username='slave'} at 127.0.0.1:57372
I20260620 23:03:59.619192 22157 tablet_service.cc:1514] Processing DeleteTablet for tablet 016b2b438b134f2f9d2ade5da8785dec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:59 UTC) from {username='slave'} at 127.0.0.1:40900
I20260620 23:03:59.620723 22027 tablet_service.cc:1514] Processing DeleteTablet for tablet 016b2b438b134f2f9d2ade5da8785dec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:03:59 UTC) from {username='slave'} at 127.0.0.1:42310
I20260620 23:03:59.621622 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 21834
I20260620 23:03:59.627218 22265 tablet_replica.cc:330] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44: stopping tablet replica
I20260620 23:03:59.628667 22265 raft_consensus.cc:2229] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44 [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:03:59.629642 22265 raft_consensus.cc:2258] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:59.630558 22264 tablet_replica.cc:330] T 016b2b438b134f2f9d2ade5da8785dec P 890631203877422d8e4542789f125191: stopping tablet replica
I20260620 23:03:59.631035 22264 raft_consensus.cc:2229] T 016b2b438b134f2f9d2ade5da8785dec P 890631203877422d8e4542789f125191 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:03:59.631695 22264 raft_consensus.cc:2258] T 016b2b438b134f2f9d2ade5da8785dec P 890631203877422d8e4542789f125191 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:59.632511 22263 tablet_replica.cc:330] T 016b2b438b134f2f9d2ade5da8785dec P a654fc35397c4506a84d487f2a1d7655: stopping tablet replica
I20260620 23:03:59.632911 22263 raft_consensus.cc:2229] T 016b2b438b134f2f9d2ade5da8785dec P a654fc35397c4506a84d487f2a1d7655 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:03:59.633350 22263 raft_consensus.cc:2258] T 016b2b438b134f2f9d2ade5da8785dec P a654fc35397c4506a84d487f2a1d7655 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:03:59.635599 22264 ts_tablet_manager.cc:1905] T 016b2b438b134f2f9d2ade5da8785dec P 890631203877422d8e4542789f125191: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:59.636109 22263 ts_tablet_manager.cc:1905] T 016b2b438b134f2f9d2ade5da8785dec P a654fc35397c4506a84d487f2a1d7655: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:59.637164 22265 ts_tablet_manager.cc:1905] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:03:59.639412 22239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:59.623508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57384 (local address 127.2.237.1:46049)
0620 23:03:59.623742 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:59.623752 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:59.628228 (+ 4476us) server_negotiation.cc:407] Connection header received
0620 23:03:59.628607 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:59.628617 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:59.628764 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:59.628904 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:59.629976 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.631700 (+ 1724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.633924 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.634608 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.634720 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:59.637672 (+ 2952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:59.637684 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:59.637710 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:59.638467 (+ 757us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:59.638527 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:59.638551 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:59.638680 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:59.638931 (+ 251us) server_negotiation.cc:299] Negotiation successful
0620 23:03:59.639046 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260620 23:03:59.640739 21956 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:03:59.643883 22263 ts_tablet_manager.cc:1918] T 016b2b438b134f2f9d2ade5da8785dec P a654fc35397c4506a84d487f2a1d7655: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:03:59.644086 22263 log.cc:1196] T 016b2b438b134f2f9d2ade5da8785dec P a654fc35397c4506a84d487f2a1d7655: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-2/wal/wals/016b2b438b134f2f9d2ade5da8785dec
I20260620 23:03:59.644730 22263 ts_tablet_manager.cc:1939] T 016b2b438b134f2f9d2ade5da8785dec P a654fc35397c4506a84d487f2a1d7655: Deleting consensus metadata
I20260620 23:03:59.646719 21762 catalog_manager.cc:4841] TS a654fc35397c4506a84d487f2a1d7655 (127.2.237.3:41293): tablet 016b2b438b134f2f9d2ade5da8785dec (table test-table [id=72017230891a4f91917146b836fddaa5]) successfully deleted
I20260620 23:03:59.647394 22265 ts_tablet_manager.cc:1918] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:03:59.647617 22265 log.cc:1196] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781996318807807-2996-0/minicluster-data/ts-1/wal/wals/016b2b438b134f2f9d2ade5da8785dec
I20260620 23:03:59.648080 22265 ts_tablet_manager.cc:1939] T 016b2b438b134f2f9d2ade5da8785dec P f8d82b9d91a144a0aea2519a88b89c44: Deleting consensus metadata
I20260620 23:03:59.649504 21759 catalog_manager.cc:4841] TS f8d82b9d91a144a0aea2519a88b89c44 (127.2.237.2:40771): tablet 016b2b438b134f2f9d2ade5da8785dec (table test-table [id=72017230891a4f91917146b836fddaa5]) successfully deleted
I20260620 23:03:59.741271 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 21834
W20260620 23:03:59.759639 21762 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.237.1:46049 (error 108)
I20260620 23:03:59.760324 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 21965
I20260620 23:03:59.761744 22236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:59.761164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:46049 (local address 127.0.0.1:57398)
0620 23:03:59.761406 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:59.761550 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
W20260620 23:03:59.762035 21762 catalog_manager.cc:4584] TS 890631203877422d8e4542789f125191 (127.2.237.1:46049): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 016b2b438b134f2f9d2ade5da8785dec: Network error: Client connection negotiation failed: client connection to 127.2.237.1:46049: connect: Connection refused (error 111)
I20260620 23:03:59.769449 22238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:59.761186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42324 (local address 127.2.237.2:40771)
0620 23:03:59.761379 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:59.761385 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:59.761497 (+ 112us) server_negotiation.cc:407] Connection header received
0620 23:03:59.761675 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:59.761681 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:59.761801 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:59.761914 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:59.762457 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.763652 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.765551 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.766199 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.766332 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:59.767619 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:59.767632 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:59.767652 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:59.768452 (+ 800us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:59.768510 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:59.768536 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:59.768646 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:59.768806 (+ 160us) server_negotiation.cc:299] Negotiation successful
0620 23:03:59.768909 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260620 23:03:59.770143 22087 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:03:59.815714 22236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:59.815163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:46049 (local address 127.0.0.1:57402)
0620 23:03:59.815402 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:59.815519 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260620 23:03:59.816875 22236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:59.816459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:46049 (local address 127.0.0.1:57416)
0620 23:03:59.816642 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:59.816730 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260620 23:03:59.877724 22236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:59.877147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:46049 (local address 127.0.0.1:57430)
0620 23:03:59.877421 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:59.877534 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260620 23:03:59.878934 22236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:59.878488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:46049 (local address 127.0.0.1:57440)
0620 23:03:59.878706 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:59.878791 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260620 23:03:59.890986 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 21965
I20260620 23:03:59.910161 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 22095
I20260620 23:03:59.917553 22235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:59.911056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40910 (local address 127.2.237.3:41293)
0620 23:03:59.911236 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0620 23:03:59.911244 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:03:59.911316 (+ 72us) server_negotiation.cc:407] Connection header received
0620 23:03:59.911563 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:03:59.911567 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:03:59.911696 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:03:59.911818 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:03:59.912283 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.912999 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.914350 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:03:59.914818 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:03:59.914953 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:03:59.916165 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:03:59.916174 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:03:59.916190 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:03:59.916774 (+ 584us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:03:59.916822 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:03:59.916841 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:03:59.916980 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0620 23:03:59.917118 (+ 138us) server_negotiation.cc:299] Negotiation successful
0620 23:03:59.917229 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260620 23:03:59.918296 22217 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:03:59.960256 22236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:59.959631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:46049 (local address 127.0.0.1:57444)
0620 23:03:59.959909 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:59.960038 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260620 23:03:59.961513 22236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:03:59.961057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:46049 (local address 127.0.0.1:57458)
0620 23:03:59.961280 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0620 23:03:59.961378 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260620 23:04:00.009756 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 22095
I20260620 23:04:00.026238 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 21744
I20260620 23:04:00.027493 21805 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:04:00.139642 22236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:00.138994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:46049 (local address 127.0.0.1:57472)
0620 23:04:00.139288 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:00.139426 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.1:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
I20260620 23:04:00.139689 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 21744
2026-06-20T23:04:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2879 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:04:00 dist-test-slave-f63b krb5kdc[22274](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:04:00 dist-test-slave-f63b krb5kdc[22274](info): set up 2 sockets
Jun 20 23:04:00 dist-test-slave-f63b krb5kdc[22274](info): commencing operation
krb5kdc: starting...
W20260620 23:04:01.250880 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 23:04:01 dist-test-slave-f63b krb5kdc[22274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:04:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:04:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
I20260620 23:04:01.522665 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:37639
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:38587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:37639
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:01.853665 22287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:01.854043 22287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:01.854207 22287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:01.861541 22287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:01.861690 22287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:01.861804 22287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:01.861898 22287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:01.861984 22287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:01.872218 22287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:38587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:37639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:37639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:01.873720 22287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:01.875183 22287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:01.887801 22292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:01.887907 22295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:01.887935 22293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:01.888635 22287 server_base.cc:1034] running on GCE node
I20260620 23:04:01.889458 22287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:01.891034 22287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:01.892278 22287 hybrid_clock.cc:648] HybridClock initialized: now 1781996641892239 us; error 53 us; skew 500 ppm
I20260620 23:04:01.892637 22287 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:04:01 dist-test-slave-f63b krb5kdc[22274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996641, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:01.895908 22287 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 23:04:01.897357 22287 webserver.cc:460] Webserver started at http://127.2.237.62:38147/ using document root <none> and password file <none>
I20260620 23:04:01.898029 22287 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:01.898130 22287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:01.898394 22287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:01.901473 22287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "1591e9c046af44fca1f9b1a65c3d1f79"
format_stamp: "Formatted at 2026-06-20 23:04:01 on dist-test-slave-f63b"
server_key: "442e18560fd81d10925ccae7c070a3d7"
server_key_iv: "aa77e53c89de8ded3a5622bd296e5b08"
server_key_version: "encryptionkey@0"
I20260620 23:04:01.902212 22287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "1591e9c046af44fca1f9b1a65c3d1f79"
format_stamp: "Formatted at 2026-06-20 23:04:01 on dist-test-slave-f63b"
server_key: "442e18560fd81d10925ccae7c070a3d7"
server_key_iv: "aa77e53c89de8ded3a5622bd296e5b08"
server_key_version: "encryptionkey@0"
I20260620 23:04:01.907889 22287 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260620 23:04:01.911634 22302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:01.913065 22287 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260620 23:04:01.913259 22287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "1591e9c046af44fca1f9b1a65c3d1f79"
format_stamp: "Formatted at 2026-06-20 23:04:01 on dist-test-slave-f63b"
server_key: "442e18560fd81d10925ccae7c070a3d7"
server_key_iv: "aa77e53c89de8ded3a5622bd296e5b08"
server_key_version: "encryptionkey@0"
I20260620 23:04:01.913487 22287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:01.952562 22287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:01.957432 22287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:01.957741 22287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:01.976950 22287 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:37639
I20260620 23:04:01.976984 22353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:37639 every 8 connection(s)
I20260620 23:04:01.978430 22287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:04:01.982250 22354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:01.987622 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 22287
I20260620 23:04:01.987821 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:04:01.988245 2996 external_mini_cluster.cc:1428] Setting key 6e04327c25f2373ab876e0cdea5a89fd
I20260620 23:04:01.993245 22354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1591e9c046af44fca1f9b1a65c3d1f79: Bootstrap starting.
I20260620 23:04:01.996981 22354 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1591e9c046af44fca1f9b1a65c3d1f79: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:01.997963 22354 log.cc:824] T 00000000000000000000000000000000 P 1591e9c046af44fca1f9b1a65c3d1f79: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:02.000854 22354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1591e9c046af44fca1f9b1a65c3d1f79: No bootstrap required, opened a new log
Jun 20 23:04:02 dist-test-slave-f63b krb5kdc[22274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:04:02.008826 22354 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1591e9c046af44fca1f9b1a65c3d1f79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1591e9c046af44fca1f9b1a65c3d1f79" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37639 } }
I20260620 23:04:02.009267 22354 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1591e9c046af44fca1f9b1a65c3d1f79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:02.009395 22354 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1591e9c046af44fca1f9b1a65c3d1f79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1591e9c046af44fca1f9b1a65c3d1f79, State: Initialized, Role: FOLLOWER
I20260620 23:04:02.010114 22354 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1591e9c046af44fca1f9b1a65c3d1f79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1591e9c046af44fca1f9b1a65c3d1f79" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37639 } }
I20260620 23:04:02.010416 22354 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1591e9c046af44fca1f9b1a65c3d1f79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:04:02.010592 22354 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1591e9c046af44fca1f9b1a65c3d1f79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:04:02.010797 22354 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1591e9c046af44fca1f9b1a65c3d1f79 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:02.011550 22357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:01.990179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48480 (local address 127.2.237.62:37639)
0620 23:04:01.991113 (+ 934us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:01.991127 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:01.991166 (+ 39us) server_negotiation.cc:407] Connection header received
0620 23:04:01.992832 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:01.992854 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:01.993212 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:01.993569 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:01.995545 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:01.997026 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:01.998614 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:01.999367 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:01.999580 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:02.003779 (+ 4199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:02.003789 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:02.003804 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:02.003846 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:02.007109 (+ 3263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:02.007830 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:02.007837 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:02.007844 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:02.008023 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:02.008432 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:02.008438 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:02.008441 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:02.009010 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:02.009335 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:02.009769 (+ 434us) server_negotiation.cc:299] Negotiation successful
0620 23:04:02.010226 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":689,"thread_start_us":391,"threads_started":1}
I20260620 23:04:02.013499 22354 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1591e9c046af44fca1f9b1a65c3d1f79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1591e9c046af44fca1f9b1a65c3d1f79" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37639 } }
I20260620 23:04:02.014025 22354 leader_election.cc:302] T 00000000000000000000000000000000 P 1591e9c046af44fca1f9b1a65c3d1f79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1591e9c046af44fca1f9b1a65c3d1f79; no voters:
I20260620 23:04:02.014714 22354 leader_election.cc:288] T 00000000000000000000000000000000 P 1591e9c046af44fca1f9b1a65c3d1f79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:04:02.014806 22359 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1591e9c046af44fca1f9b1a65c3d1f79 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:02.015619 22359 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1591e9c046af44fca1f9b1a65c3d1f79 [term 1 LEADER]: Becoming Leader. State: Replica: 1591e9c046af44fca1f9b1a65c3d1f79, State: Running, Role: LEADER
I20260620 23:04:02.016109 22359 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1591e9c046af44fca1f9b1a65c3d1f79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1591e9c046af44fca1f9b1a65c3d1f79" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37639 } }
I20260620 23:04:02.016835 22354 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1591e9c046af44fca1f9b1a65c3d1f79 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:02.018947 22361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1591e9c046af44fca1f9b1a65c3d1f79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1591e9c046af44fca1f9b1a65c3d1f79. Latest consensus state: current_term: 1 leader_uuid: "1591e9c046af44fca1f9b1a65c3d1f79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1591e9c046af44fca1f9b1a65c3d1f79" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37639 } } }
I20260620 23:04:02.019462 22361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1591e9c046af44fca1f9b1a65c3d1f79 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:02.019024 22360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1591e9c046af44fca1f9b1a65c3d1f79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1591e9c046af44fca1f9b1a65c3d1f79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1591e9c046af44fca1f9b1a65c3d1f79" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37639 } } }
I20260620 23:04:02.019670 22360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1591e9c046af44fca1f9b1a65c3d1f79 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:02.020856 22367 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:04:02.027717 22367 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:04:02.037276 22367 catalog_manager.cc:1347] Generated new cluster ID: 2288f0315fd54dcba5b1c983a810f8f3
I20260620 23:04:02.037405 22367 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:04:02.065351 22367 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:04:02.066846 22367 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:04:02.076439 22367 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1591e9c046af44fca1f9b1a65c3d1f79: Generated new TSK 0
I20260620 23:04:02.077280 22367 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.237.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
Entry for principal kudu/127.2.237.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab.
I20260620 23:04:02.151474 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:0
--local_ip_for_outbound_sockets=127.2.237.1
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:37639
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:38587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:02.496625 22380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:02.496923 22380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:02.497030 22380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:02.504463 22380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:02.504577 22380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:02.504691 22380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 23:04:02.515748 22380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:38587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.1.keytab
--principal=kudu/127.2.237.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:37639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:02.517174 22380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:02.518785 22380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:02.532656 22385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:02.532747 22388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:02.532775 22386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:02.533263 22380 server_base.cc:1034] running on GCE node
I20260620 23:04:02.533932 22380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:02.535027 22380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:02.536252 22380 hybrid_clock.cc:648] HybridClock initialized: now 1781996642536200 us; error 49 us; skew 500 ppm
I20260620 23:04:02.536588 22380 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:04:02 dist-test-slave-f63b krb5kdc[22274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996642, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:02.539997 22380 init.cc:377] Logged in from keytab as kudu/127.2.237.1@KRBTEST.COM (short username kudu)
I20260620 23:04:02.541543 22380 webserver.cc:460] Webserver started at http://127.2.237.1:42877/ using document root <none> and password file <none>
I20260620 23:04:02.542232 22380 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:02.542340 22380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:02.542678 22380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:02.545639 22380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-0/data/instance:
uuid: "2714e10cc48c46e7ab4cce06cda13f30"
format_stamp: "Formatted at 2026-06-20 23:04:02 on dist-test-slave-f63b"
server_key: "874a9caf409cbe0597802dc8723012ef"
server_key_iv: "b4e1571cd1269005c2ee9b92fa7a14f4"
server_key_version: "encryptionkey@0"
I20260620 23:04:02.546381 22380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance:
uuid: "2714e10cc48c46e7ab4cce06cda13f30"
format_stamp: "Formatted at 2026-06-20 23:04:02 on dist-test-slave-f63b"
server_key: "874a9caf409cbe0597802dc8723012ef"
server_key_iv: "b4e1571cd1269005c2ee9b92fa7a14f4"
server_key_version: "encryptionkey@0"
I20260620 23:04:02.551810 22380 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260620 23:04:02.555444 22395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:02.556959 22380 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 23:04:02.557106 22380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "2714e10cc48c46e7ab4cce06cda13f30"
format_stamp: "Formatted at 2026-06-20 23:04:02 on dist-test-slave-f63b"
server_key: "874a9caf409cbe0597802dc8723012ef"
server_key_iv: "b4e1571cd1269005c2ee9b92fa7a14f4"
server_key_version: "encryptionkey@0"
I20260620 23:04:02.557286 22380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:02.577574 22380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:02.581811 22380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:02.582063 22380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:02.583093 22380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:04:02.584424 22380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:04:02.584498 22380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:02.584585 22380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:04:02.584630 22380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:02.619094 22380 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:36803
I20260620 23:04:02.619144 22507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:36803 every 8 connection(s)
I20260620 23:04:02.620424 22380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 23:04:02.621181 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 22380
I20260620 23:04:02.621403 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance
I20260620 23:04:02.621970 2996 external_mini_cluster.cc:1428] Setting key ad60b6856ab6942fbdaa07e2581a38c5
Jun 20 23:04:02 dist-test-slave-f63b krb5kdc[22274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996642, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.1@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:04:02.642462 22510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:02.623170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:57719 (local address 127.2.237.62:37639)
0620 23:04:02.623660 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:02.623666 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:02.624935 (+ 1269us) server_negotiation.cc:407] Connection header received
0620 23:04:02.626914 (+ 1979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:02.626922 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:02.627010 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:02.627128 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:02.628742 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:02.629947 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:02.631624 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:02.632142 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:02.632247 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:02.636526 (+ 4279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:02.636531 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:02.636534 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:02.636571 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:02.639273 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:02.640042 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:02.640053 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:02.640055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:02.640179 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:02.640651 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:02.640660 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:02.640662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:02.640926 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:02.641096 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:02.641857 (+ 761us) server_negotiation.cc:299] Negotiation successful
0620 23:04:02.642087 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":240,"threads_started":1}
I20260620 23:04:02.643572 22511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:02.623861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37639 (local address 127.2.237.1:57719)
0620 23:04:02.624728 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:02.624802 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:02.626656 (+ 1854us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:02.627331 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:02.627339 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:02.627973 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:02.628559 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:02.628577 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:02.630095 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:02.630098 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:02.631472 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:02.631484 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:02.632416 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:02.632424 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:02.632775 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:02.633598 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:02.633628 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:02.636263 (+ 2635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:02.639435 (+ 3172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:02.639445 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:02.639465 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:02.639905 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:02.640306 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:02.640312 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:02.640317 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:02.640519 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:02.641068 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:02.641076 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:02.641448 (+ 372us) client_negotiation.cc:769] Sending connection context
0620 23:04:02.641739 (+ 291us) client_negotiation.cc:240] Negotiation successful
0620 23:04:02.642318 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":373,"threads_started":1}
I20260620 23:04:02.645382 22508 heartbeater.cc:347] Connected to a master server at 127.2.237.62:37639
I20260620 23:04:02.645701 22508 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.2.237.2@KRBTEST.COM; defaulting to no policy
I20260620 23:04:02.646513 22508 heartbeater.cc:511] Master 127.2.237.62:37639 requested a full tablet report, sending...
I20260620 23:04:02.648622 22319 ts_manager.cc:194] Registered new tserver with Master: 2714e10cc48c46e7ab4cce06cda13f30 (127.2.237.1:36803)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.2@KRBTEST.COM" created.
I20260620 23:04:02.650167 22319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.1@KRBTEST.COM'} at 127.2.237.1:57719
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
Entry for principal kudu/127.2.237.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab.
I20260620 23:04:02.677506 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:0
--local_ip_for_outbound_sockets=127.2.237.2
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:37639
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:38587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:03.008589 22514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:03.008893 22514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:03.009061 22514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:03.016713 22514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:03.016889 22514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:03.017087 22514 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 23:04:03.027520 22514 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:38587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.2.keytab
--principal=kudu/127.2.237.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:37639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:03.028932 22514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:03.030375 22514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:03.044190 22520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:03.044229 22519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:03.044359 22522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:03.044983 22514 server_base.cc:1034] running on GCE node
I20260620 23:04:03.045650 22514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:03.046777 22514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:03.048015 22514 hybrid_clock.cc:648] HybridClock initialized: now 1781996643047945 us; error 66 us; skew 500 ppm
I20260620 23:04:03.048345 22514 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:04:03 dist-test-slave-f63b krb5kdc[22274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996643, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:03.060272 22514 init.cc:377] Logged in from keytab as kudu/127.2.237.2@KRBTEST.COM (short username kudu)
I20260620 23:04:03.062155 22514 webserver.cc:460] Webserver started at http://127.2.237.2:41695/ using document root <none> and password file <none>
I20260620 23:04:03.063086 22514 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:03.063231 22514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:03.063557 22514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:03.066634 22514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-1/data/instance:
uuid: "b29b4766f8bc47d0a30fcdd6524b74e3"
format_stamp: "Formatted at 2026-06-20 23:04:03 on dist-test-slave-f63b"
server_key: "e9145944a9d0f877117f543c6fbc021d"
server_key_iv: "cc05a15ce0d8c48de11b946aca767063"
server_key_version: "encryptionkey@0"
I20260620 23:04:03.067456 22514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance:
uuid: "b29b4766f8bc47d0a30fcdd6524b74e3"
format_stamp: "Formatted at 2026-06-20 23:04:03 on dist-test-slave-f63b"
server_key: "e9145944a9d0f877117f543c6fbc021d"
server_key_iv: "cc05a15ce0d8c48de11b946aca767063"
server_key_version: "encryptionkey@0"
I20260620 23:04:03.072595 22514 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260620 23:04:03.076071 22529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:03.077514 22514 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 23:04:03.077693 22514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "b29b4766f8bc47d0a30fcdd6524b74e3"
format_stamp: "Formatted at 2026-06-20 23:04:03 on dist-test-slave-f63b"
server_key: "e9145944a9d0f877117f543c6fbc021d"
server_key_iv: "cc05a15ce0d8c48de11b946aca767063"
server_key_version: "encryptionkey@0"
I20260620 23:04:03.077906 22514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:03.110291 22514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:03.115061 22514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:03.115381 22514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:03.116437 22514 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:04:03.118021 22514 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:04:03.118124 22514 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:03.118269 22514 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:04:03.118350 22514 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:03.153890 22514 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:46559
I20260620 23:04:03.153990 22641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:46559 every 8 connection(s)
I20260620 23:04:03.155380 22514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 23:04:03.164211 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 22514
I20260620 23:04:03.164446 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance
I20260620 23:04:03.165062 2996 external_mini_cluster.cc:1428] Setting key c33e736e83fad25d3b557e1645962837
Jun 20 23:04:03 dist-test-slave-f63b krb5kdc[22274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996643, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.2@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:04:03.178129 22644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:03.158111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:46431 (local address 127.2.237.62:37639)
0620 23:04:03.158616 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:03.158623 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:03.159579 (+ 956us) server_negotiation.cc:407] Connection header received
0620 23:04:03.161718 (+ 2139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:03.161726 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:03.161848 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:03.161957 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:03.163799 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.164651 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.166825 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.167481 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.167598 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:03.171847 (+ 4249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:03.171852 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:03.171855 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:03.171894 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:03.174872 (+ 2978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:03.175646 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:03.175651 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:03.175654 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:03.175794 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:03.176249 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:03.176259 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:03.176261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:03.176530 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:03.176691 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:03.177452 (+ 761us) server_negotiation.cc:299] Negotiation successful
0620 23:04:03.177735 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":220,"threads_started":1}
I20260620 23:04:03.179208 22645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:03.158665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37639 (local address 127.2.237.2:46431)
0620 23:04:03.159404 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:03.159446 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:03.161387 (+ 1941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:03.162271 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:03.162287 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:03.162954 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:03.163619 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:03.163639 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.164901 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.164907 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:03.166695 (+ 1788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:03.166708 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.167738 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.167746 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:03.168057 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:03.168784 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:03.168820 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:03.171604 (+ 2784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:03.175022 (+ 3418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:03.175032 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:03.175051 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:03.175500 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:03.175912 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:03.175918 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:03.175922 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:03.176121 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:03.176660 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:03.176667 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:03.177037 (+ 370us) client_negotiation.cc:769] Sending connection context
0620 23:04:03.177361 (+ 324us) client_negotiation.cc:240] Negotiation successful
0620 23:04:03.177959 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":299,"threads_started":1}
I20260620 23:04:03.180729 22642 heartbeater.cc:347] Connected to a master server at 127.2.237.62:37639
I20260620 23:04:03.181067 22642 heartbeater.cc:464] Registering TS with master...
I20260620 23:04:03.182019 22642 heartbeater.cc:511] Master 127.2.237.62:37639 requested a full tablet report, sending...
I20260620 23:04:03.183799 22319 ts_manager.cc:194] Registered new tserver with Master: b29b4766f8bc47d0a30fcdd6524b74e3 (127.2.237.2:46559)
I20260620 23:04:03.184702 22319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.2@KRBTEST.COM'} at 127.2.237.2:46431
WARNING: no policy specified for kudu/127.2.237.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
Entry for principal kudu/127.2.237.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab.
I20260620 23:04:03.222004 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:0
--local_ip_for_outbound_sockets=127.2.237.3
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:37639
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:38587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:03.553359 22648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:03.553728 22648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:03.553910 22648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:03.561621 22648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:03.561806 22648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:03.562027 22648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 23:04:03.572319 22648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:38587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.3.keytab
--principal=kudu/127.2.237.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:37639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:03.573706 22648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:03.575328 22648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:03.589432 22653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:03.589366 22654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:03.589463 22656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:03.590497 22648 server_base.cc:1034] running on GCE node
I20260620 23:04:03.591217 22648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:03.592221 22648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:03.593453 22648 hybrid_clock.cc:648] HybridClock initialized: now 1781996643593387 us; error 68 us; skew 500 ppm
I20260620 23:04:03.593782 22648 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:04:03 dist-test-slave-f63b krb5kdc[22274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996643, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:03.597158 22648 init.cc:377] Logged in from keytab as kudu/127.2.237.3@KRBTEST.COM (short username kudu)
I20260620 23:04:03.598893 22648 webserver.cc:460] Webserver started at http://127.2.237.3:35799/ using document root <none> and password file <none>
I20260620 23:04:03.599648 22648 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:03.599808 22648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:03.600139 22648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:03.603278 22648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-2/data/instance:
uuid: "51e4f23ede1b45bda013998bff8dafa2"
format_stamp: "Formatted at 2026-06-20 23:04:03 on dist-test-slave-f63b"
server_key: "273e6cf58ba4449a41bad4d052901e05"
server_key_iv: "d0eb3ffb38823b74c66d8058db5431fa"
server_key_version: "encryptionkey@0"
I20260620 23:04:03.604120 22648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance:
uuid: "51e4f23ede1b45bda013998bff8dafa2"
format_stamp: "Formatted at 2026-06-20 23:04:03 on dist-test-slave-f63b"
server_key: "273e6cf58ba4449a41bad4d052901e05"
server_key_iv: "d0eb3ffb38823b74c66d8058db5431fa"
server_key_version: "encryptionkey@0"
I20260620 23:04:03.609484 22648 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260620 23:04:03.613153 22663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:03.614634 22648 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260620 23:04:03.614833 22648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "51e4f23ede1b45bda013998bff8dafa2"
format_stamp: "Formatted at 2026-06-20 23:04:03 on dist-test-slave-f63b"
server_key: "273e6cf58ba4449a41bad4d052901e05"
server_key_iv: "d0eb3ffb38823b74c66d8058db5431fa"
server_key_version: "encryptionkey@0"
I20260620 23:04:03.615074 22648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:03.627455 22648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:03.631680 22648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:03.631991 22648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:03.633026 22648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:04:03.634415 22648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:04:03.634521 22648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:03.634684 22648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:04:03.634760 22648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:03.653283 22508 heartbeater.cc:503] Master 127.2.237.62:37639 was elected leader, sending a full tablet report...
I20260620 23:04:03.670156 22648 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:40631
I20260620 23:04:03.670197 22775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:40631 every 8 connection(s)
I20260620 23:04:03.671599 22648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 23:04:03.677937 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 22648
I20260620 23:04:03.678153 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance
I20260620 23:04:03.678668 2996 external_mini_cluster.cc:1428] Setting key 0d1446dfa18e6eb06b90fefa78ba342f
Jun 20 23:04:03 dist-test-slave-f63b krb5kdc[22274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996643, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.3@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:04:03.694316 22644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:03.674467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:36537 (local address 127.2.237.62:37639)
0620 23:04:03.674698 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:03.674704 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:03.675954 (+ 1250us) server_negotiation.cc:407] Connection header received
0620 23:04:03.677957 (+ 2003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:03.677962 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:03.678087 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:03.678236 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:03.680291 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.681569 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.683453 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.683869 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.683991 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:03.687572 (+ 3581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:03.687578 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:03.687581 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:03.687618 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:03.690661 (+ 3043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:03.691468 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:03.691473 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:03.691475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:03.691601 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:03.692119 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:03.692123 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:03.692126 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:03.692513 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:03.692687 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:03.693733 (+ 1046us) server_negotiation.cc:299] Negotiation successful
0620 23:04:03.693987 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260620 23:04:03.695356 22778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:03.675022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37639 (local address 127.2.237.3:36537)
0620 23:04:03.675758 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:03.675804 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:03.677604 (+ 1800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:03.678500 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:03.678509 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:03.679236 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:03.680087 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:03.680115 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.681727 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.681731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:03.683293 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:03.683310 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.684081 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.684085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:03.684281 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:03.684943 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:03.684965 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:03.687287 (+ 2322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:03.690819 (+ 3532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:03.690826 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:03.690840 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:03.691317 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:03.691707 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:03.691710 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:03.691713 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:03.691933 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:03.692676 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:03.692684 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:03.693188 (+ 504us) client_negotiation.cc:769] Sending connection context
0620 23:04:03.693530 (+ 342us) client_negotiation.cc:240] Negotiation successful
0620 23:04:03.694111 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":304,"threads_started":1}
I20260620 23:04:03.696808 22776 heartbeater.cc:347] Connected to a master server at 127.2.237.62:37639
I20260620 23:04:03.697160 22776 heartbeater.cc:464] Registering TS with master...
I20260620 23:04:03.697911 22776 heartbeater.cc:511] Master 127.2.237.62:37639 requested a full tablet report, sending...
I20260620 23:04:03.699491 22319 ts_manager.cc:194] Registered new tserver with Master: 51e4f23ede1b45bda013998bff8dafa2 (127.2.237.3:40631)
I20260620 23:04:03.700330 22319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.237.3@KRBTEST.COM'} at 127.2.237.3:36537
I20260620 23:04:03.707414 2996 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:04:03.725071 22644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:03.711578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48488 (local address 127.2.237.62:37639)
0620 23:04:03.711767 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:03.711774 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:03.712193 (+ 419us) server_negotiation.cc:407] Connection header received
0620 23:04:03.712414 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:03.712418 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:03.712492 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:03.712606 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:03.714039 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.714835 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.716205 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.716616 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.716753 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:03.719486 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:03.719492 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:03.719503 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:03.719539 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:03.722057 (+ 2518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:03.722849 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:03.722855 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:03.722858 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:03.723002 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:03.723402 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:03.723413 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:03.723416 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:03.723814 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:03.724111 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:03.724473 (+ 362us) server_negotiation.cc:299] Negotiation successful
0620 23:04:03.724718 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260620 23:04:03.728938 22319 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48488:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:04:03.736074 22319 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:04:03.763720 22789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:03.749125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45086 (local address 127.2.237.3:40631)
0620 23:04:03.754561 (+ 5436us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:03.754605 (+ 44us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:03.754655 (+ 50us) server_negotiation.cc:407] Connection header received
0620 23:04:03.754802 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:03.754815 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:03.755142 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:03.756163 (+ 1021us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:04:03.756531 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.758264 (+ 1733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.761090 (+ 2826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.762439 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.762607 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:03.762870 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:03.763237 (+ 367us) server_negotiation.cc:299] Negotiation successful
0620 23:04:03.763401 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5187,"thread_start_us":4995,"threads_started":1}
I20260620 23:04:03.763767 22790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:03.749032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:40631 (local address 127.0.0.1:45086)
0620 23:04:03.751953 (+ 2921us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:03.751973 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:03.752077 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:03.755433 (+ 3356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:03.755442 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:03.755477 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:04:03.755928 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:03.755942 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.758418 (+ 2476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.758428 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:03.760900 (+ 2472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:03.760920 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.762661 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.762668 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:03.762864 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:03.762923 (+ 59us) client_negotiation.cc:769] Sending connection context
0620 23:04:03.763086 (+ 163us) client_negotiation.cc:240] Negotiation successful
0620 23:04:03.763294 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2789,"spinlock_wait_cycles":6016,"thread_start_us":2656,"threads_started":1}
I20260620 23:04:03.764431 22791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:03.749016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:46559 (local address 127.0.0.1:39934)
0620 23:04:03.751095 (+ 2079us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:03.751147 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:03.751325 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:03.756328 (+ 5003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:03.756334 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:03.756364 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:04:03.756844 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:03.756860 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.758964 (+ 2104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.758974 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:03.761162 (+ 2188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:03.761175 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.763479 (+ 2304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.763487 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:03.763694 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:03.763738 (+ 44us) client_negotiation.cc:769] Sending connection context
0620 23:04:03.763828 (+ 90us) client_negotiation.cc:240] Negotiation successful
0620 23:04:03.763981 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1832,"mutex_wait_us":31,"thread_start_us":1673,"threads_started":1}
I20260620 23:04:03.765079 22788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:03.749077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39934 (local address 127.2.237.2:46559)
0620 23:04:03.755656 (+ 6579us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:03.755665 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:03.755694 (+ 29us) server_negotiation.cc:407] Connection header received
0620 23:04:03.755810 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:03.755822 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:03.756080 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:03.757071 (+ 991us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:04:03.757312 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.758832 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.761339 (+ 2507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.763226 (+ 1887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.764343 (+ 1117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:03.764537 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:03.764608 (+ 71us) server_negotiation.cc:299] Negotiation successful
0620 23:04:03.764757 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6418,"thread_start_us":6296,"threads_started":1}
I20260620 23:04:03.766394 22792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:03.752063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:36803 (local address 127.0.0.1:40548)
0620 23:04:03.755392 (+ 3329us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:03.755410 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:03.755547 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:03.756089 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:03.756093 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:03.756119 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:04:03.756596 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:03.756609 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.758207 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.758213 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:03.763141 (+ 4928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:03.763158 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.765550 (+ 2392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.765556 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:03.765716 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:03.765748 (+ 32us) client_negotiation.cc:769] Sending connection context
0620 23:04:03.765834 (+ 86us) client_negotiation.cc:240] Negotiation successful
0620 23:04:03.765965 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3180,"mutex_wait_us":1993,"spinlock_wait_cycles":11392,"thread_start_us":2990,"threads_started":1}
I20260620 23:04:03.767143 22793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:03.752686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40548 (local address 127.2.237.1:36803)
0620 23:04:03.754865 (+ 2179us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:03.754877 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:03.755625 (+ 748us) server_negotiation.cc:407] Connection header received
0620 23:04:03.755729 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:03.755734 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:03.755906 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:03.756093 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:04:03.756801 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.758073 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.763380 (+ 5307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.765326 (+ 1946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.765473 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:03.765701 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:03.766635 (+ 934us) server_negotiation.cc:299] Negotiation successful
0620 23:04:03.766812 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1978,"thread_start_us":1801,"threads_started":1}
I20260620 23:04:03.768594 22577 tablet_service.cc:1467] Processing CreateTablet for tablet fc022aadb97742e695845cfcc939ece4 (DEFAULT_TABLE table=test-table [id=5b2a4991e9f34b0fa221f350ad19d05e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:04:03.768514 22711 tablet_service.cc:1467] Processing CreateTablet for tablet fc022aadb97742e695845cfcc939ece4 (DEFAULT_TABLE table=test-table [id=5b2a4991e9f34b0fa221f350ad19d05e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:04:03.769814 22577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc022aadb97742e695845cfcc939ece4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:03.770100 22711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc022aadb97742e695845cfcc939ece4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:03.770651 22443 tablet_service.cc:1467] Processing CreateTablet for tablet fc022aadb97742e695845cfcc939ece4 (DEFAULT_TABLE table=test-table [id=5b2a4991e9f34b0fa221f350ad19d05e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:04:03.772089 22443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc022aadb97742e695845cfcc939ece4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:03.780793 22794 tablet_bootstrap.cc:492] T fc022aadb97742e695845cfcc939ece4 P b29b4766f8bc47d0a30fcdd6524b74e3: Bootstrap starting.
I20260620 23:04:03.784693 22794 tablet_bootstrap.cc:654] T fc022aadb97742e695845cfcc939ece4 P b29b4766f8bc47d0a30fcdd6524b74e3: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:03.785473 22795 tablet_bootstrap.cc:492] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2: Bootstrap starting.
I20260620 23:04:03.786144 22794 log.cc:824] T fc022aadb97742e695845cfcc939ece4 P b29b4766f8bc47d0a30fcdd6524b74e3: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:03.789069 22795 tablet_bootstrap.cc:654] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:03.789646 22794 tablet_bootstrap.cc:492] T fc022aadb97742e695845cfcc939ece4 P b29b4766f8bc47d0a30fcdd6524b74e3: No bootstrap required, opened a new log
I20260620 23:04:03.789997 22794 ts_tablet_manager.cc:1397] T fc022aadb97742e695845cfcc939ece4 P b29b4766f8bc47d0a30fcdd6524b74e3: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260620 23:04:03.790503 22795 log.cc:824] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:03.790808 22797 tablet_bootstrap.cc:492] T fc022aadb97742e695845cfcc939ece4 P 2714e10cc48c46e7ab4cce06cda13f30: Bootstrap starting.
I20260620 23:04:03.794299 22797 tablet_bootstrap.cc:654] T fc022aadb97742e695845cfcc939ece4 P 2714e10cc48c46e7ab4cce06cda13f30: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:03.794477 22795 tablet_bootstrap.cc:492] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2: No bootstrap required, opened a new log
I20260620 23:04:03.794783 22795 ts_tablet_manager.cc:1397] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260620 23:04:03.795516 22797 log.cc:824] T fc022aadb97742e695845cfcc939ece4 P 2714e10cc48c46e7ab4cce06cda13f30: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:03.798233 22797 tablet_bootstrap.cc:492] T fc022aadb97742e695845cfcc939ece4 P 2714e10cc48c46e7ab4cce06cda13f30: No bootstrap required, opened a new log
I20260620 23:04:03.798516 22797 ts_tablet_manager.cc:1397] T fc022aadb97742e695845cfcc939ece4 P 2714e10cc48c46e7ab4cce06cda13f30: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.004s
I20260620 23:04:03.800587 22794 raft_consensus.cc:348] T fc022aadb97742e695845cfcc939ece4 P b29b4766f8bc47d0a30fcdd6524b74e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29b4766f8bc47d0a30fcdd6524b74e3" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46559 } } peers { permanent_uuid: "51e4f23ede1b45bda013998bff8dafa2" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40631 } } peers { permanent_uuid: "2714e10cc48c46e7ab4cce06cda13f30" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 36803 } }
I20260620 23:04:03.801083 22794 raft_consensus.cc:374] T fc022aadb97742e695845cfcc939ece4 P b29b4766f8bc47d0a30fcdd6524b74e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:03.801196 22794 raft_consensus.cc:729] T fc022aadb97742e695845cfcc939ece4 P b29b4766f8bc47d0a30fcdd6524b74e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b29b4766f8bc47d0a30fcdd6524b74e3, State: Initialized, Role: FOLLOWER
I20260620 23:04:03.801832 22794 consensus_queue.cc:260] T fc022aadb97742e695845cfcc939ece4 P b29b4766f8bc47d0a30fcdd6524b74e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29b4766f8bc47d0a30fcdd6524b74e3" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46559 } } peers { permanent_uuid: "51e4f23ede1b45bda013998bff8dafa2" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40631 } } peers { permanent_uuid: "2714e10cc48c46e7ab4cce06cda13f30" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 36803 } }
I20260620 23:04:03.803043 22642 heartbeater.cc:503] Master 127.2.237.62:37639 was elected leader, sending a full tablet report...
I20260620 23:04:03.803646 22794 ts_tablet_manager.cc:1428] T fc022aadb97742e695845cfcc939ece4 P b29b4766f8bc47d0a30fcdd6524b74e3: Time spent starting tablet: real 0.013s user 0.002s sys 0.010s
I20260620 23:04:03.805054 22795 raft_consensus.cc:348] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29b4766f8bc47d0a30fcdd6524b74e3" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46559 } } peers { permanent_uuid: "51e4f23ede1b45bda013998bff8dafa2" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40631 } } peers { permanent_uuid: "2714e10cc48c46e7ab4cce06cda13f30" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 36803 } }
I20260620 23:04:03.805505 22795 raft_consensus.cc:374] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:03.805611 22795 raft_consensus.cc:729] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51e4f23ede1b45bda013998bff8dafa2, State: Initialized, Role: FOLLOWER
I20260620 23:04:03.806209 22795 consensus_queue.cc:260] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29b4766f8bc47d0a30fcdd6524b74e3" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46559 } } peers { permanent_uuid: "51e4f23ede1b45bda013998bff8dafa2" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40631 } } peers { permanent_uuid: "2714e10cc48c46e7ab4cce06cda13f30" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 36803 } }
I20260620 23:04:03.807390 22776 heartbeater.cc:503] Master 127.2.237.62:37639 was elected leader, sending a full tablet report...
I20260620 23:04:03.808108 22795 ts_tablet_manager.cc:1428] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260620 23:04:03.809465 22797 raft_consensus.cc:348] T fc022aadb97742e695845cfcc939ece4 P 2714e10cc48c46e7ab4cce06cda13f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29b4766f8bc47d0a30fcdd6524b74e3" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46559 } } peers { permanent_uuid: "51e4f23ede1b45bda013998bff8dafa2" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40631 } } peers { permanent_uuid: "2714e10cc48c46e7ab4cce06cda13f30" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 36803 } }
I20260620 23:04:03.810061 22797 raft_consensus.cc:374] T fc022aadb97742e695845cfcc939ece4 P 2714e10cc48c46e7ab4cce06cda13f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:03.810184 22797 raft_consensus.cc:729] T fc022aadb97742e695845cfcc939ece4 P 2714e10cc48c46e7ab4cce06cda13f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2714e10cc48c46e7ab4cce06cda13f30, State: Initialized, Role: FOLLOWER
I20260620 23:04:03.810842 22797 consensus_queue.cc:260] T fc022aadb97742e695845cfcc939ece4 P 2714e10cc48c46e7ab4cce06cda13f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29b4766f8bc47d0a30fcdd6524b74e3" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46559 } } peers { permanent_uuid: "51e4f23ede1b45bda013998bff8dafa2" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40631 } } peers { permanent_uuid: "2714e10cc48c46e7ab4cce06cda13f30" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 36803 } }
I20260620 23:04:03.812599 22797 ts_tablet_manager.cc:1428] T fc022aadb97742e695845cfcc939ece4 P 2714e10cc48c46e7ab4cce06cda13f30: Time spent starting tablet: real 0.014s user 0.007s sys 0.006s
I20260620 23:04:03.837985 22801 raft_consensus.cc:482] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:04:03.838274 22801 raft_consensus.cc:504] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29b4766f8bc47d0a30fcdd6524b74e3" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46559 } } peers { permanent_uuid: "51e4f23ede1b45bda013998bff8dafa2" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40631 } } peers { permanent_uuid: "2714e10cc48c46e7ab4cce06cda13f30" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 36803 } }
I20260620 23:04:03.840234 22801 leader_election.cc:288] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b29b4766f8bc47d0a30fcdd6524b74e3 (127.2.237.2:46559), 2714e10cc48c46e7ab4cce06cda13f30 (127.2.237.1:36803)
I20260620 23:04:03.846969 22788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:03.840787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:53705 (local address 127.2.237.2:46559)
0620 23:04:03.841060 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:03.841068 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:03.841145 (+ 77us) server_negotiation.cc:407] Connection header received
0620 23:04:03.841417 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:03.841425 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:03.841508 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:03.841629 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:04:03.842170 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.843104 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.844902 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.846052 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.846166 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:03.846244 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:03.846649 (+ 405us) server_negotiation.cc:299] Negotiation successful
0620 23:04:03.846753 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260620 23:04:03.847015 22778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:03.840611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:46559 (local address 127.2.237.3:53705)
0620 23:04:03.841007 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:03.841037 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:03.841240 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:03.841683 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:03.841690 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:03.841726 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:04:03.842011 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:03.842025 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.843234 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.843240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:03.844715 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:03.844726 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.846226 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.846231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:03.846424 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:03.846458 (+ 34us) client_negotiation.cc:769] Sending connection context
0620 23:04:03.846558 (+ 100us) client_negotiation.cc:240] Negotiation successful
0620 23:04:03.846719 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211}
I20260620 23:04:03.847916 22803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:03.840659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:36803 (local address 127.2.237.3:36201)
0620 23:04:03.841540 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:03.841558 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:03.841675 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:03.842047 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:03.842050 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:03.842069 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:04:03.842470 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:03.842483 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.843741 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.843745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:03.845564 (+ 1819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:03.845580 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.847176 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.847199 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:03.847385 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:03.847423 (+ 38us) client_negotiation.cc:769] Sending connection context
0620 23:04:03.847548 (+ 125us) client_negotiation.cc:240] Negotiation successful
0620 23:04:03.847657 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":771,"thread_start_us":544,"threads_started":1}
I20260620 23:04:03.848078 22793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:03.840838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:36201 (local address 127.2.237.1:36803)
0620 23:04:03.841023 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:03.841029 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:03.841663 (+ 634us) server_negotiation.cc:407] Connection header received
0620 23:04:03.841814 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:03.841822 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:03.841903 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:03.842010 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:04:03.842697 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.843568 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.845770 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.846941 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.847079 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:03.847199 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:03.847613 (+ 414us) server_negotiation.cc:299] Negotiation successful
0620 23:04:03.847761 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260620 23:04:03.848573 22597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc022aadb97742e695845cfcc939ece4" candidate_uuid: "51e4f23ede1b45bda013998bff8dafa2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b29b4766f8bc47d0a30fcdd6524b74e3" is_pre_election: true
I20260620 23:04:03.849198 22597 raft_consensus.cc:2454] T fc022aadb97742e695845cfcc939ece4 P b29b4766f8bc47d0a30fcdd6524b74e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51e4f23ede1b45bda013998bff8dafa2 in term 0.
I20260620 23:04:03.849077 22463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc022aadb97742e695845cfcc939ece4" candidate_uuid: "51e4f23ede1b45bda013998bff8dafa2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2714e10cc48c46e7ab4cce06cda13f30" is_pre_election: true
I20260620 23:04:03.849689 22463 raft_consensus.cc:2454] T fc022aadb97742e695845cfcc939ece4 P 2714e10cc48c46e7ab4cce06cda13f30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51e4f23ede1b45bda013998bff8dafa2 in term 0.
I20260620 23:04:03.849987 22666 leader_election.cc:302] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51e4f23ede1b45bda013998bff8dafa2, b29b4766f8bc47d0a30fcdd6524b74e3; no voters:
I20260620 23:04:03.850406 22801 raft_consensus.cc:2781] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:04:03.850553 22801 raft_consensus.cc:482] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:04:03.850689 22801 raft_consensus.cc:3037] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:03.852876 22801 raft_consensus.cc:504] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29b4766f8bc47d0a30fcdd6524b74e3" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46559 } } peers { permanent_uuid: "51e4f23ede1b45bda013998bff8dafa2" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40631 } } peers { permanent_uuid: "2714e10cc48c46e7ab4cce06cda13f30" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 36803 } }
I20260620 23:04:03.853521 22801 leader_election.cc:288] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2 [CANDIDATE]: Term 1 election: Requested vote from peers b29b4766f8bc47d0a30fcdd6524b74e3 (127.2.237.2:46559), 2714e10cc48c46e7ab4cce06cda13f30 (127.2.237.1:36803)
I20260620 23:04:03.853957 22597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc022aadb97742e695845cfcc939ece4" candidate_uuid: "51e4f23ede1b45bda013998bff8dafa2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b29b4766f8bc47d0a30fcdd6524b74e3"
I20260620 23:04:03.853976 22463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc022aadb97742e695845cfcc939ece4" candidate_uuid: "51e4f23ede1b45bda013998bff8dafa2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2714e10cc48c46e7ab4cce06cda13f30"
I20260620 23:04:03.854171 22463 raft_consensus.cc:3037] T fc022aadb97742e695845cfcc939ece4 P 2714e10cc48c46e7ab4cce06cda13f30 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:03.854180 22597 raft_consensus.cc:3037] T fc022aadb97742e695845cfcc939ece4 P b29b4766f8bc47d0a30fcdd6524b74e3 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:03.856369 22463 raft_consensus.cc:2454] T fc022aadb97742e695845cfcc939ece4 P 2714e10cc48c46e7ab4cce06cda13f30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51e4f23ede1b45bda013998bff8dafa2 in term 1.
I20260620 23:04:03.856743 22597 raft_consensus.cc:2454] T fc022aadb97742e695845cfcc939ece4 P b29b4766f8bc47d0a30fcdd6524b74e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51e4f23ede1b45bda013998bff8dafa2 in term 1.
I20260620 23:04:03.856778 22665 leader_election.cc:302] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2714e10cc48c46e7ab4cce06cda13f30, 51e4f23ede1b45bda013998bff8dafa2; no voters:
I20260620 23:04:03.857088 22801 raft_consensus.cc:2781] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:03.857538 22801 raft_consensus.cc:686] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2 [term 1 LEADER]: Becoming Leader. State: Replica: 51e4f23ede1b45bda013998bff8dafa2, State: Running, Role: LEADER
I20260620 23:04:03.858090 22801 consensus_queue.cc:237] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29b4766f8bc47d0a30fcdd6524b74e3" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46559 } } peers { permanent_uuid: "51e4f23ede1b45bda013998bff8dafa2" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40631 } } peers { permanent_uuid: "2714e10cc48c46e7ab4cce06cda13f30" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 36803 } }
I20260620 23:04:03.862987 22318 catalog_manager.cc:5495] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51e4f23ede1b45bda013998bff8dafa2 (127.2.237.3). New cstate: current_term: 1 leader_uuid: "51e4f23ede1b45bda013998bff8dafa2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29b4766f8bc47d0a30fcdd6524b74e3" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51e4f23ede1b45bda013998bff8dafa2" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 40631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2714e10cc48c46e7ab4cce06cda13f30" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 36803 } health_report { overall_health: UNKNOWN } } }
W20260620 23:04:03.872944 22509 tablet.cc:2369] T fc022aadb97742e695845cfcc939ece4 P 2714e10cc48c46e7ab4cce06cda13f30: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:04:03.885520 22789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:03.880074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45102 (local address 127.2.237.3:40631)
0620 23:04:03.880259 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:03.880264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:03.880367 (+ 103us) server_negotiation.cc:407] Connection header received
0620 23:04:03.880541 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:03.880544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:03.880630 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:03.880813 (+ 183us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:03.881235 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.882044 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.883514 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.883903 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.884006 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:03.884441 (+ 435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:03.884954 (+ 513us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:03.885061 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:03.885163 (+ 102us) server_negotiation.cc:299] Negotiation successful
0620 23:04:03.885228 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 23:04:03.894200 22463 raft_consensus.cc:1261] T fc022aadb97742e695845cfcc939ece4 P 2714e10cc48c46e7ab4cce06cda13f30 [term 1 FOLLOWER]: Refusing update from remote peer 51e4f23ede1b45bda013998bff8dafa2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:04:03.894200 22597 raft_consensus.cc:1261] T fc022aadb97742e695845cfcc939ece4 P b29b4766f8bc47d0a30fcdd6524b74e3 [term 1 FOLLOWER]: Refusing update from remote peer 51e4f23ede1b45bda013998bff8dafa2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:04:03.895517 22804 consensus_queue.cc:1035] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b29b4766f8bc47d0a30fcdd6524b74e3" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 46559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:04:03.895927 22801 consensus_queue.cc:1035] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2714e10cc48c46e7ab4cce06cda13f30" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 36803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 23:04:03.908263 22643 tablet.cc:2369] T fc022aadb97742e695845cfcc939ece4 P b29b4766f8bc47d0a30fcdd6524b74e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:04:03.914471 22809 mvcc.cc:205] Tried to move back new op lower bound from 7299058253378727936 to 7299058253247254528. Current Snapshot: MvccSnapshot[applied={T|T < 7299058253378727936}]
I20260620 23:04:03.926707 22319 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48488:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:04:03.927027 22319 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48488:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:04:03.930598 22319 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1591e9c046af44fca1f9b1a65c3d1f79: Sending DeleteTablet for 3 replicas of tablet fc022aadb97742e695845cfcc939ece4
I20260620 23:04:03.931999 22577 tablet_service.cc:1514] Processing DeleteTablet for tablet fc022aadb97742e695845cfcc939ece4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:04:03 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:39934
I20260620 23:04:03.932078 22443 tablet_service.cc:1514] Processing DeleteTablet for tablet fc022aadb97742e695845cfcc939ece4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:04:03 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:40548
I20260620 23:04:03.933000 22816 tablet_replica.cc:330] T fc022aadb97742e695845cfcc939ece4 P b29b4766f8bc47d0a30fcdd6524b74e3: stopping tablet replica
I20260620 23:04:03.933210 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 22380
I20260620 23:04:03.933126 22711 tablet_service.cc:1514] Processing DeleteTablet for tablet fc022aadb97742e695845cfcc939ece4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:04:03 UTC) from {username='kudu', principal='kudu/127.2.237.62@KRBTEST.COM'} at 127.0.0.1:45086
I20260620 23:04:03.934507 22816 raft_consensus.cc:2229] T fc022aadb97742e695845cfcc939ece4 P b29b4766f8bc47d0a30fcdd6524b74e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:04:03.935734 22816 raft_consensus.cc:2258] T fc022aadb97742e695845cfcc939ece4 P b29b4766f8bc47d0a30fcdd6524b74e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:04:03.938155 22816 ts_tablet_manager.cc:1905] T fc022aadb97742e695845cfcc939ece4 P b29b4766f8bc47d0a30fcdd6524b74e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:04:03.944152 22817 tablet_replica.cc:330] T fc022aadb97742e695845cfcc939ece4 P 2714e10cc48c46e7ab4cce06cda13f30: stopping tablet replica
I20260620 23:04:03.944931 22817 raft_consensus.cc:2229] T fc022aadb97742e695845cfcc939ece4 P 2714e10cc48c46e7ab4cce06cda13f30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:04:03.945681 22817 raft_consensus.cc:2258] T fc022aadb97742e695845cfcc939ece4 P 2714e10cc48c46e7ab4cce06cda13f30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:04:03.949857 22818 tablet_replica.cc:330] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2: stopping tablet replica
I20260620 23:04:03.950057 22817 ts_tablet_manager.cc:1905] T fc022aadb97742e695845cfcc939ece4 P 2714e10cc48c46e7ab4cce06cda13f30: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:04:03.950361 22818 raft_consensus.cc:2229] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2 [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:04:03.951161 22818 raft_consensus.cc:2258] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:04:03.952215 22816 ts_tablet_manager.cc:1918] T fc022aadb97742e695845cfcc939ece4 P b29b4766f8bc47d0a30fcdd6524b74e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:04:03.952468 22816 log.cc:1196] T fc022aadb97742e695845cfcc939ece4 P b29b4766f8bc47d0a30fcdd6524b74e3: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-1/wal/wals/fc022aadb97742e695845cfcc939ece4
I20260620 23:04:03.953075 22816 ts_tablet_manager.cc:1939] T fc022aadb97742e695845cfcc939ece4 P b29b4766f8bc47d0a30fcdd6524b74e3: Deleting consensus metadata
I20260620 23:04:03.955065 22818 ts_tablet_manager.cc:1905] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:04:03.955040 22305 catalog_manager.cc:4841] TS b29b4766f8bc47d0a30fcdd6524b74e3 (127.2.237.2:46559): tablet fc022aadb97742e695845cfcc939ece4 (table test-table [id=5b2a4991e9f34b0fa221f350ad19d05e]) successfully deleted
I20260620 23:04:03.957823 22817 ts_tablet_manager.cc:1918] T fc022aadb97742e695845cfcc939ece4 P 2714e10cc48c46e7ab4cce06cda13f30: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:04:03.958024 22817 log.cc:1196] T fc022aadb97742e695845cfcc939ece4 P 2714e10cc48c46e7ab4cce06cda13f30: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-0/wal/wals/fc022aadb97742e695845cfcc939ece4
I20260620 23:04:03.958698 22817 ts_tablet_manager.cc:1939] T fc022aadb97742e695845cfcc939ece4 P 2714e10cc48c46e7ab4cce06cda13f30: Deleting consensus metadata
I20260620 23:04:03.960793 22304 catalog_manager.cc:4841] TS 2714e10cc48c46e7ab4cce06cda13f30 (127.2.237.1:36803): tablet fc022aadb97742e695845cfcc939ece4 (table test-table [id=5b2a4991e9f34b0fa221f350ad19d05e]) successfully deleted
Jun 20 23:04:03 dist-test-slave-f63b krb5kdc[22274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.1@KRBTEST.COM
I20260620 23:04:03.963038 22818 ts_tablet_manager.cc:1918] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:04:03.963212 22818 log.cc:1196] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781996318807807-2996-0/minicluster-data/ts-2/wal/wals/fc022aadb97742e695845cfcc939ece4
I20260620 23:04:03.963768 22818 ts_tablet_manager.cc:1939] T fc022aadb97742e695845cfcc939ece4 P 51e4f23ede1b45bda013998bff8dafa2: Deleting consensus metadata
I20260620 23:04:03.965553 22304 catalog_manager.cc:4841] TS 51e4f23ede1b45bda013998bff8dafa2 (127.2.237.3:40631): tablet fc022aadb97742e695845cfcc939ece4 (table test-table [id=5b2a4991e9f34b0fa221f350ad19d05e]) successfully deleted
I20260620 23:04:03.969274 22793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:03.935064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40552 (local address 127.2.237.1:36803)
0620 23:04:03.935251 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:03.935257 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:03.943725 (+ 8468us) server_negotiation.cc:407] Connection header received
0620 23:04:03.944049 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:03.944055 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:03.944154 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:03.944231 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:03.946081 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.947336 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.951410 (+ 4074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:03.952080 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:03.952188 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:03.962671 (+ 10483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:03.962684 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:03.962698 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:03.962735 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:03.966253 (+ 3518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:03.967089 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:03.967096 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:03.967099 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:03.967231 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:03.967670 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:03.967680 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:03.967682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:03.968069 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:03.968378 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:03.968702 (+ 324us) server_negotiation.cc:299] Negotiation successful
0620 23:04:03.968933 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260620 23:04:03.969957 22503 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:04:04.079850 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 22380
I20260620 23:04:04.098477 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 22514
Jun 20 23:04:04 dist-test-slave-f63b krb5kdc[22274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.2@KRBTEST.COM
I20260620 23:04:04.117651 22788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:04.099452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39936 (local address 127.2.237.2:46559)
0620 23:04:04.099621 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:04.099626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:04.099667 (+ 41us) server_negotiation.cc:407] Connection header received
0620 23:04:04.099935 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:04.099944 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:04.100025 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:04.100129 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:04.101973 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:04.102807 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:04.104773 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:04.105173 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:04.105297 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:04.110562 (+ 5265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:04.110644 (+ 82us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:04.110661 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:04.110704 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:04.114287 (+ 3583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:04.115181 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:04.115191 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:04.115195 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:04.115349 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:04.115790 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:04.115797 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:04.115801 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:04.116289 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:04.116628 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:04.117003 (+ 375us) server_negotiation.cc:299] Negotiation successful
0620 23:04:04.117264 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260620 23:04:04.118549 22637 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:04:04.209826 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 22514
I20260620 23:04:04.234054 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 22648
Jun 20 23:04:04 dist-test-slave-f63b krb5kdc[22274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.3@KRBTEST.COM
I20260620 23:04:04.253158 22789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:04.235009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45116 (local address 127.2.237.3:40631)
0620 23:04:04.235184 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:04.235188 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:04.235206 (+ 18us) server_negotiation.cc:407] Connection header received
0620 23:04:04.235411 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:04.235415 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:04.235482 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:04.235590 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:04.237325 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:04.238178 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:04.240208 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:04.240656 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:04.240757 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:04.246489 (+ 5732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:04.246501 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:04.246516 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:04.246550 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:04.249827 (+ 3277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:04.250908 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:04.250916 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:04.250918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:04.251092 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:04.251540 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:04.251544 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:04.251546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:04.251923 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:04.252162 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:04.252596 (+ 434us) server_negotiation.cc:299] Negotiation successful
0620 23:04:04.252758 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260620 23:04:04.254046 22771 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:04:04.378639 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 22648
I20260620 23:04:04.399987 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 22287
I20260620 23:04:04.401049 22349 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:04:04.514474 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 22287
2026-06-20T23:04:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4375 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-20T23:04:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:04:04Z Disabled control of system clock
I20260620 23:04:04.591656 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:34489
--webserver_interface=127.2.237.62
--webserver_port=0
--builtin_ntp_servers=127.2.237.20:41495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:34489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 23:04:04.930260 22827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:04.930617 22827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:04.930745 22827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:04.938067 22827 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:04.938187 22827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:04.938225 22827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:04.938274 22827 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:04.938323 22827 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:04.947999 22827 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:41495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:34489
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:34489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:04.949290 22827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:04.951089 22827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:04.963341 22832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:04.963441 22833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:04.963462 22835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:04.964708 22827 server_base.cc:1034] running on GCE node
I20260620 23:04:04.965629 22827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:04.967276 22827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:04.968541 22827 hybrid_clock.cc:648] HybridClock initialized: now 1781996644968501 us; error 57 us; skew 500 ppm
I20260620 23:04:04.968935 22827 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 23:04:04.971392 22827 webserver.cc:460] Webserver started at http://127.2.237.62:39723/ using document root <none> and password file <none>
I20260620 23:04:04.972132 22827 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:04.972276 22827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:04.972611 22827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:04.975786 22827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "22548efa420646d9ae0137a042eb935a"
format_stamp: "Formatted at 2026-06-20 23:04:04 on dist-test-slave-f63b"
server_key: "1a2efe5f982d76fea9f6440b1770c56e"
server_key_iv: "7e0d5b72302c7dc8838c6a846b79f606"
server_key_version: "encryptionkey@0"
I20260620 23:04:04.976650 22827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "22548efa420646d9ae0137a042eb935a"
format_stamp: "Formatted at 2026-06-20 23:04:04 on dist-test-slave-f63b"
server_key: "1a2efe5f982d76fea9f6440b1770c56e"
server_key_iv: "7e0d5b72302c7dc8838c6a846b79f606"
server_key_version: "encryptionkey@0"
I20260620 23:04:04.982077 22827 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260620 23:04:04.985908 22841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:04.987479 22827 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 23:04:04.987699 22827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "22548efa420646d9ae0137a042eb935a"
format_stamp: "Formatted at 2026-06-20 23:04:04 on dist-test-slave-f63b"
server_key: "1a2efe5f982d76fea9f6440b1770c56e"
server_key_iv: "7e0d5b72302c7dc8838c6a846b79f606"
server_key_version: "encryptionkey@0"
I20260620 23:04:04.987942 22827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:05.000618 22827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:05.001518 22827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:05.001842 22827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:05.021202 22827 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:34489
I20260620 23:04:05.021210 22892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:34489 every 8 connection(s)
I20260620 23:04:05.022647 22827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:04:05.026301 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 22827
I20260620 23:04:05.026489 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:04:05.026649 22893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:05.027045 2996 external_mini_cluster.cc:1428] Setting key 3004d475b2075cd483dc6e213d5aef44
I20260620 23:04:05.041919 22893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22548efa420646d9ae0137a042eb935a: Bootstrap starting.
I20260620 23:04:05.042948 22896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:05.029046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60894 (local address 127.2.237.62:34489)
0620 23:04:05.030191 (+ 1145us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:05.030208 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:05.030265 (+ 57us) server_negotiation.cc:407] Connection header received
0620 23:04:05.032098 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:05.032133 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:05.032791 (+ 658us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:05.033285 (+ 494us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:05.033872 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:05.035938 (+ 2066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:05.037432 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:05.038477 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:05.038736 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:05.039963 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:05.039972 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:05.039989 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:05.040691 (+ 702us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:05.040790 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:05.040814 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:05.040934 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:05.041223 (+ 289us) server_negotiation.cc:299] Negotiation successful
0620 23:04:05.041543 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":375,"threads_started":1}
I20260620 23:04:05.045712 22893 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22548efa420646d9ae0137a042eb935a: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:05.046785 22893 log.cc:824] T 00000000000000000000000000000000 P 22548efa420646d9ae0137a042eb935a: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:05.049680 22893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22548efa420646d9ae0137a042eb935a: No bootstrap required, opened a new log
I20260620 23:04:05.057049 22893 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22548efa420646d9ae0137a042eb935a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22548efa420646d9ae0137a042eb935a" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 34489 } }
I20260620 23:04:05.057422 22893 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22548efa420646d9ae0137a042eb935a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:05.057488 22893 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22548efa420646d9ae0137a042eb935a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22548efa420646d9ae0137a042eb935a, State: Initialized, Role: FOLLOWER
I20260620 23:04:05.058053 22893 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22548efa420646d9ae0137a042eb935a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22548efa420646d9ae0137a042eb935a" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 34489 } }
I20260620 23:04:05.058261 22893 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22548efa420646d9ae0137a042eb935a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:04:05.058364 22893 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22548efa420646d9ae0137a042eb935a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:04:05.058485 22893 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22548efa420646d9ae0137a042eb935a [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:05.060609 22893 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22548efa420646d9ae0137a042eb935a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22548efa420646d9ae0137a042eb935a" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 34489 } }
I20260620 23:04:05.061062 22893 leader_election.cc:302] T 00000000000000000000000000000000 P 22548efa420646d9ae0137a042eb935a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22548efa420646d9ae0137a042eb935a; no voters:
I20260620 23:04:05.061587 22893 leader_election.cc:288] T 00000000000000000000000000000000 P 22548efa420646d9ae0137a042eb935a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:04:05.061757 22898 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22548efa420646d9ae0137a042eb935a [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:05.062633 22898 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22548efa420646d9ae0137a042eb935a [term 1 LEADER]: Becoming Leader. State: Replica: 22548efa420646d9ae0137a042eb935a, State: Running, Role: LEADER
I20260620 23:04:05.063288 22898 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22548efa420646d9ae0137a042eb935a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22548efa420646d9ae0137a042eb935a" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 34489 } }
I20260620 23:04:05.064023 22893 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22548efa420646d9ae0137a042eb935a [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:05.066360 22900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22548efa420646d9ae0137a042eb935a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22548efa420646d9ae0137a042eb935a. Latest consensus state: current_term: 1 leader_uuid: "22548efa420646d9ae0137a042eb935a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22548efa420646d9ae0137a042eb935a" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 34489 } } }
I20260620 23:04:05.066481 22899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22548efa420646d9ae0137a042eb935a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22548efa420646d9ae0137a042eb935a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22548efa420646d9ae0137a042eb935a" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 34489 } } }
I20260620 23:04:05.066893 22899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22548efa420646d9ae0137a042eb935a [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:05.066776 22900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22548efa420646d9ae0137a042eb935a [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:05.067809 22904 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:04:05.073984 22904 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:04:05.083307 22904 catalog_manager.cc:1347] Generated new cluster ID: c3351c4dc0cc4f87ae692679936d753d
I20260620 23:04:05.083464 22904 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:04:05.095144 22904 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:04:05.096524 22904 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:04:05.107659 22904 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22548efa420646d9ae0137a042eb935a: Generated new TSK 0
I20260620 23:04:05.108613 22904 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:04:05.114403 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.1:0
--local_ip_for_outbound_sockets=127.2.237.1
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:34489
--builtin_ntp_servers=127.2.237.20:41495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260620 23:04:05.467170 22917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:05.467562 22917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:05.467720 22917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:05.475363 22917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:05.475476 22917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:05.475648 22917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.1
I20260620 23:04:05.485080 22917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:41495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.237.1
--webserver_port=0
--tserver_master_addrs=127.2.237.62:34489
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.1
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:05.486500 22917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:05.488382 22917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:05.502240 22925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:05.502271 22923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:05.502394 22922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:05.503181 22917 server_base.cc:1034] running on GCE node
I20260620 23:04:05.503971 22917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:05.505074 22917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:05.506325 22917 hybrid_clock.cc:648] HybridClock initialized: now 1781996645506263 us; error 55 us; skew 500 ppm
I20260620 23:04:05.506744 22917 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 23:04:05.509262 22917 webserver.cc:460] Webserver started at http://127.2.237.1:39305/ using document root <none> and password file <none>
I20260620 23:04:05.510053 22917 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:05.510198 22917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:05.510552 22917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:05.513849 22917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-0/data/instance:
uuid: "d64af67b60864d03871f3d2c92868cb9"
format_stamp: "Formatted at 2026-06-20 23:04:05 on dist-test-slave-f63b"
server_key: "21502d99ab63e03b008eaf45767e1480"
server_key_iv: "239e2628881d8d99ed8e68614ace9d0d"
server_key_version: "encryptionkey@0"
I20260620 23:04:05.514829 22917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance:
uuid: "d64af67b60864d03871f3d2c92868cb9"
format_stamp: "Formatted at 2026-06-20 23:04:05 on dist-test-slave-f63b"
server_key: "21502d99ab63e03b008eaf45767e1480"
server_key_iv: "239e2628881d8d99ed8e68614ace9d0d"
server_key_version: "encryptionkey@0"
I20260620 23:04:05.520385 22917 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260620 23:04:05.524444 22931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:05.525938 22917 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 23:04:05.526139 22917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-0/wal
uuid: "d64af67b60864d03871f3d2c92868cb9"
format_stamp: "Formatted at 2026-06-20 23:04:05 on dist-test-slave-f63b"
server_key: "21502d99ab63e03b008eaf45767e1480"
server_key_iv: "239e2628881d8d99ed8e68614ace9d0d"
server_key_version: "encryptionkey@0"
I20260620 23:04:05.526389 22917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:05.547320 22917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:05.548202 22917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:05.548486 22917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:05.549479 22917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:04:05.551017 22917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:04:05.551138 22917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:05.551267 22917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:04:05.551357 22917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:05.586038 22917 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.1:33629
I20260620 23:04:05.586104 23043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.1:33629 every 8 connection(s)
I20260620 23:04:05.587484 22917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-0/data/info.pb
I20260620 23:04:05.594161 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 22917
I20260620 23:04:05.594300 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-0/wal/instance
I20260620 23:04:05.594869 2996 external_mini_cluster.cc:1428] Setting key 0b7a07b38149ca112aa4856f5c543eaa
I20260620 23:04:05.597517 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.2:0
--local_ip_for_outbound_sockets=127.2.237.2
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:34489
--builtin_ntp_servers=127.2.237.20:41495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 23:04:05.602512 23046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:05.590431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.1:35609 (local address 127.2.237.62:34489)
0620 23:04:05.590987 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:05.590991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:05.592211 (+ 1220us) server_negotiation.cc:407] Connection header received
0620 23:04:05.594650 (+ 2439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:05.594662 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:05.594744 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:05.594886 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:05.596312 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:05.597303 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:05.599267 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:05.599711 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:05.599873 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:05.601460 (+ 1587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:05.601466 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:05.601469 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:05.601488 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:05.601527 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:05.601540 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:05.601646 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:05.601936 (+ 290us) server_negotiation.cc:299] Negotiation successful
0620 23:04:05.602043 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":339,"threads_started":1}
I20260620 23:04:05.603678 23047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:05.590976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:34489 (local address 127.2.237.1:35609)
0620 23:04:05.592014 (+ 1038us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:05.592059 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:05.594141 (+ 2082us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:05.595161 (+ 1020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:05.595171 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:05.595284 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:05.596118 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:05.596139 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:05.597483 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:05.597488 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:05.599097 (+ 1609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:05.599112 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:05.599973 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:05.599983 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:05.600245 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:05.601088 (+ 843us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:04:05.601091 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:05.601136 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:04:05.601228 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:05.601646 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:05.601660 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:05.601671 (+ 11us) client_negotiation.cc:769] Sending connection context
0620 23:04:05.601912 (+ 241us) client_negotiation.cc:240] Negotiation successful
0620 23:04:05.602469 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":341,"threads_started":1}
I20260620 23:04:05.605327 23044 heartbeater.cc:347] Connected to a master server at 127.2.237.62:34489
I20260620 23:04:05.605638 23044 heartbeater.cc:464] Registering TS with master...
I20260620 23:04:05.606352 23044 heartbeater.cc:511] Master 127.2.237.62:34489 requested a full tablet report, sending...
I20260620 23:04:05.608932 22858 ts_manager.cc:194] Registered new tserver with Master: d64af67b60864d03871f3d2c92868cb9 (127.2.237.1:33629)
I20260620 23:04:05.610670 22858 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.237.1:35609
W20260620 23:04:05.929803 23048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:05.930130 23048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:05.930269 23048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:05.937865 23048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:05.937980 23048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:05.938160 23048 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.2
I20260620 23:04:05.947548 23048 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:41495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.237.2
--webserver_port=0
--tserver_master_addrs=127.2.237.62:34489
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.2
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:05.948869 23048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:05.951022 23048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:05.966867 23054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:05.966930 23056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:05.966820 23053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:05.967765 23048 server_base.cc:1034] running on GCE node
I20260620 23:04:05.968537 23048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:05.969667 23048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:05.970897 23048 hybrid_clock.cc:648] HybridClock initialized: now 1781996645970838 us; error 46 us; skew 500 ppm
I20260620 23:04:05.971235 23048 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 23:04:05.973637 23048 webserver.cc:460] Webserver started at http://127.2.237.2:46799/ using document root <none> and password file <none>
I20260620 23:04:05.974373 23048 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:05.974509 23048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:05.974884 23048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:05.977802 23048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-1/data/instance:
uuid: "9bb35e7fdbaa4b68846341fe9d06c873"
format_stamp: "Formatted at 2026-06-20 23:04:05 on dist-test-slave-f63b"
server_key: "950b938af0fa60c2fef89919273b1b04"
server_key_iv: "58e021715411156c9600c5ed7b61e5a9"
server_key_version: "encryptionkey@0"
I20260620 23:04:05.978721 23048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance:
uuid: "9bb35e7fdbaa4b68846341fe9d06c873"
format_stamp: "Formatted at 2026-06-20 23:04:05 on dist-test-slave-f63b"
server_key: "950b938af0fa60c2fef89919273b1b04"
server_key_iv: "58e021715411156c9600c5ed7b61e5a9"
server_key_version: "encryptionkey@0"
I20260620 23:04:05.984246 23048 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260620 23:04:05.987854 23062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:05.989246 23048 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 23:04:05.989437 23048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-1/wal
uuid: "9bb35e7fdbaa4b68846341fe9d06c873"
format_stamp: "Formatted at 2026-06-20 23:04:05 on dist-test-slave-f63b"
server_key: "950b938af0fa60c2fef89919273b1b04"
server_key_iv: "58e021715411156c9600c5ed7b61e5a9"
server_key_version: "encryptionkey@0"
I20260620 23:04:05.989666 23048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:06.014387 23048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:06.015322 23048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:06.015611 23048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:06.016615 23048 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:04:06.018133 23048 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:04:06.018260 23048 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:06.018401 23048 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:04:06.018492 23048 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:06.061306 23048 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.2:32947
I20260620 23:04:06.061401 23174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.2:32947 every 8 connection(s)
I20260620 23:04:06.062738 23048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-1/data/info.pb
I20260620 23:04:06.064224 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 23048
I20260620 23:04:06.064429 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-1/wal/instance
I20260620 23:04:06.064939 2996 external_mini_cluster.cc:1428] Setting key bf21b9a0dad04ae8d4d2b3330d11312e
I20260620 23:04:06.068109 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.237.3:0
--local_ip_for_outbound_sockets=127.2.237.3
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:34489
--builtin_ntp_servers=127.2.237.20:41495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260620 23:04:06.076484 23046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:06.066096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:60407 (local address 127.2.237.62:34489)
0620 23:04:06.066304 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:06.066309 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:06.067942 (+ 1633us) server_negotiation.cc:407] Connection header received
0620 23:04:06.070024 (+ 2082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:06.070030 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:06.070122 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:06.070233 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:06.071414 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.072114 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.073669 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.074058 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.074196 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:06.075625 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:06.075630 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:06.075633 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:06.075651 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:06.075683 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:06.075695 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:06.075790 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:06.076062 (+ 272us) server_negotiation.cc:299] Negotiation successful
0620 23:04:06.076158 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260620 23:04:06.077489 23177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:06.066767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:34489 (local address 127.2.237.2:60407)
0620 23:04:06.067777 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:06.067823 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:06.069711 (+ 1888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:06.070487 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:06.070495 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:06.070648 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:06.071219 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:06.071235 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.072271 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.072277 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:06.073514 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:06.073525 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.074282 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.074290 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:06.074523 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:06.075295 (+ 772us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:04:06.075298 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:06.075342 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:04:06.075421 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:06.075801 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:06.075813 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:06.075823 (+ 10us) client_negotiation.cc:769] Sending connection context
0620 23:04:06.076013 (+ 190us) client_negotiation.cc:240] Negotiation successful
0620 23:04:06.076415 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":330,"threads_started":1}
I20260620 23:04:06.079146 23175 heartbeater.cc:347] Connected to a master server at 127.2.237.62:34489
I20260620 23:04:06.079463 23175 heartbeater.cc:464] Registering TS with master...
I20260620 23:04:06.080189 23175 heartbeater.cc:511] Master 127.2.237.62:34489 requested a full tablet report, sending...
I20260620 23:04:06.081740 22858 ts_manager.cc:194] Registered new tserver with Master: 9bb35e7fdbaa4b68846341fe9d06c873 (127.2.237.2:32947)
I20260620 23:04:06.083055 22858 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.237.2:60407
W20260620 23:04:06.397132 23178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:06.397423 23178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:06.397508 23178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:06.406700 23178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:06.406810 23178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:06.406916 23178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.237.3
I20260620 23:04:06.416044 23178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:41495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.237.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.237.3
--webserver_port=0
--tserver_master_addrs=127.2.237.62:34489
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.237.3
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:06.417333 23178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:06.419121 23178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:06.432183 23183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:06.432240 23184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:06.432212 23186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:06.433423 23178 server_base.cc:1034] running on GCE node
I20260620 23:04:06.434127 23178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:06.435241 23178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:06.436477 23178 hybrid_clock.cc:648] HybridClock initialized: now 1781996646436415 us; error 47 us; skew 500 ppm
I20260620 23:04:06.436834 23178 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260620 23:04:06.439263 23178 webserver.cc:460] Webserver started at http://127.2.237.3:34019/ using document root <none> and password file <none>
I20260620 23:04:06.440013 23178 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:06.440101 23178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:06.440457 23178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:06.443410 23178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-2/data/instance:
uuid: "ecbfea6397c34614a014989c88eff37c"
format_stamp: "Formatted at 2026-06-20 23:04:06 on dist-test-slave-f63b"
server_key: "0f70feb3c8720c0a04ca3a6cd333e38d"
server_key_iv: "ce22c783b2ef71a60113d6bcde79f273"
server_key_version: "encryptionkey@0"
I20260620 23:04:06.444200 23178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance:
uuid: "ecbfea6397c34614a014989c88eff37c"
format_stamp: "Formatted at 2026-06-20 23:04:06 on dist-test-slave-f63b"
server_key: "0f70feb3c8720c0a04ca3a6cd333e38d"
server_key_iv: "ce22c783b2ef71a60113d6bcde79f273"
server_key_version: "encryptionkey@0"
I20260620 23:04:06.449368 23178 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260620 23:04:06.452880 23192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:06.454370 23178 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260620 23:04:06.454612 23178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-2/wal
uuid: "ecbfea6397c34614a014989c88eff37c"
format_stamp: "Formatted at 2026-06-20 23:04:06 on dist-test-slave-f63b"
server_key: "0f70feb3c8720c0a04ca3a6cd333e38d"
server_key_iv: "ce22c783b2ef71a60113d6bcde79f273"
server_key_version: "encryptionkey@0"
I20260620 23:04:06.454854 23178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:06.478361 23178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:06.479285 23178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:06.479599 23178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:06.480615 23178 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260620 23:04:06.482165 23178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260620 23:04:06.482285 23178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:06.482496 23178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260620 23:04:06.482609 23178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:06.517011 23178 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.3:38465
I20260620 23:04:06.517105 23304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.3:38465 every 8 connection(s)
I20260620 23:04:06.518355 23178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-2/data/info.pb
I20260620 23:04:06.524710 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 23178
I20260620 23:04:06.524878 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-2/wal/instance
I20260620 23:04:06.525307 2996 external_mini_cluster.cc:1428] Setting key 255ad499e25826202ee01046f919c9a7
I20260620 23:04:06.531140 23046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:06.521112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.3:54691 (local address 127.2.237.62:34489)
0620 23:04:06.521311 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:06.521315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:06.522687 (+ 1372us) server_negotiation.cc:407] Connection header received
0620 23:04:06.524478 (+ 1791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:06.524486 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:06.524579 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:06.524701 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:06.525906 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.526838 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.528574 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.528999 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.529093 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:06.530392 (+ 1299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:06.530414 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:06.530418 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:06.530434 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:06.530465 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:06.530477 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:06.530542 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:06.530831 (+ 289us) server_negotiation.cc:299] Negotiation successful
0620 23:04:06.530905 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260620 23:04:06.532195 23307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:06.521609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:34489 (local address 127.2.237.3:54691)
0620 23:04:06.522516 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:06.522558 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:06.524196 (+ 1638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:06.524966 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:06.524980 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:06.525086 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:06.525730 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:06.525747 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.527012 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.527019 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:06.528439 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:06.528450 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.529188 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.529193 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:06.529445 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:06.530098 (+ 653us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0620 23:04:06.530101 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:06.530140 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0620 23:04:06.530219 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:06.530624 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:06.530638 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:06.530653 (+ 15us) client_negotiation.cc:769] Sending connection context
0620 23:04:06.530815 (+ 162us) client_negotiation.cc:240] Negotiation successful
0620 23:04:06.531095 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":276,"threads_started":1}
I20260620 23:04:06.533579 23305 heartbeater.cc:347] Connected to a master server at 127.2.237.62:34489
I20260620 23:04:06.533825 23305 heartbeater.cc:464] Registering TS with master...
I20260620 23:04:06.534511 23305 heartbeater.cc:511] Master 127.2.237.62:34489 requested a full tablet report, sending...
I20260620 23:04:06.535863 22858 ts_manager.cc:194] Registered new tserver with Master: ecbfea6397c34614a014989c88eff37c (127.2.237.3:38465)
I20260620 23:04:06.536688 22858 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.237.3:54691
I20260620 23:04:06.540385 2996 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260620 23:04:06.549939 23046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:06.544378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50840 (local address 127.2.237.62:34489)
0620 23:04:06.544539 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:06.544543 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:06.544997 (+ 454us) server_negotiation.cc:407] Connection header received
0620 23:04:06.545128 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:06.545132 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:06.545196 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:06.545276 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:06.545699 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.546359 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.547764 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.548195 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.548310 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:06.549363 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:06.549368 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:06.549371 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:06.549388 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:06.549419 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:06.549431 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:06.549506 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:06.549644 (+ 138us) server_negotiation.cc:299] Negotiation successful
0620 23:04:06.549715 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260620 23:04:06.553730 22858 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50840:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260620 23:04:06.560814 22858 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260620 23:04:06.587651 23317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:06.572912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:33629 (local address 127.0.0.1:55542)
0620 23:04:06.576154 (+ 3242us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:06.576184 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:06.576355 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:06.579137 (+ 2782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:06.579144 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:06.579186 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:04:06.579666 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:06.579681 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.582192 (+ 2511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.582202 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:06.584958 (+ 2756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:06.584975 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.586521 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.586528 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:06.586803 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:06.586867 (+ 64us) client_negotiation.cc:769] Sending connection context
0620 23:04:06.587029 (+ 162us) client_negotiation.cc:240] Negotiation successful
0620 23:04:06.587244 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2779,"mutex_wait_us":24,"thread_start_us":2611,"threads_started":1}
I20260620 23:04:06.587679 23319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:06.573179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55542 (local address 127.2.237.1:33629)
0620 23:04:06.575498 (+ 2319us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:06.575510 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:06.576494 (+ 984us) server_negotiation.cc:407] Connection header received
0620 23:04:06.576915 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:06.576928 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:06.578498 (+ 1570us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:06.578821 (+ 323us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:04:06.579937 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.581984 (+ 2047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.585157 (+ 3173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.586271 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.586497 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:06.586838 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:06.587085 (+ 247us) server_negotiation.cc:299] Negotiation successful
0620 23:04:06.587258 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2124,"thread_start_us":1970,"threads_started":1}
I20260620 23:04:06.591056 23320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:06.573723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53326 (local address 127.2.237.2:32947)
0620 23:04:06.574986 (+ 1263us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:06.575000 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:06.577102 (+ 2102us) server_negotiation.cc:407] Connection header received
0620 23:04:06.577307 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:06.577320 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:06.577618 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:06.577853 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:04:06.579467 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.581127 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.588040 (+ 6913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.589513 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.589651 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:06.589863 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:06.590547 (+ 684us) server_negotiation.cc:299] Negotiation successful
0620 23:04:06.590740 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1037,"thread_start_us":868,"threads_started":1}
I20260620 23:04:06.591753 23318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:06.573134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.2:32947 (local address 127.0.0.1:53326)
0620 23:04:06.576830 (+ 3696us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:06.576847 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:06.576971 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:06.578650 (+ 1679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:06.578662 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:06.578705 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:04:06.579195 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:06.579210 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.581358 (+ 2148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.581367 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:06.585044 (+ 3677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:06.585055 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.590115 (+ 5060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.590121 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:06.590314 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:06.590359 (+ 45us) client_negotiation.cc:769] Sending connection context
0620 23:04:06.591248 (+ 889us) client_negotiation.cc:240] Negotiation successful
0620 23:04:06.591420 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3538,"mutex_wait_us":867,"thread_start_us":2579,"threads_started":1}
I20260620 23:04:06.592597 22979 tablet_service.cc:1467] Processing CreateTablet for tablet 2bbe19c8ecf5406f874f46af7ddcc5ac (DEFAULT_TABLE table=test-table [id=8c298bb832ee47e7a3e682cb3da8425d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:04:06.594973 23321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:06.576384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:38465 (local address 127.0.0.1:56024)
0620 23:04:06.583680 (+ 7296us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:06.583694 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:06.583856 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:06.584871 (+ 1015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:06.584879 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:06.584902 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:04:06.587132 (+ 2230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:06.587144 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.589095 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.589100 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:06.592241 (+ 3141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:06.592256 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.594071 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.594077 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:06.594254 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:06.594290 (+ 36us) client_negotiation.cc:769] Sending connection context
0620 23:04:06.594368 (+ 78us) client_negotiation.cc:240] Negotiation successful
0620 23:04:06.594599 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7189,"thread_start_us":7070,"threads_started":1}
I20260620 23:04:06.594996 23322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:06.576629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56024 (local address 127.2.237.3:38465)
0620 23:04:06.583338 (+ 6709us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:06.583344 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:06.583819 (+ 475us) server_negotiation.cc:407] Connection header received
0620 23:04:06.584044 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:06.584052 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:06.584358 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:06.584563 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:04:06.587346 (+ 2783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.588929 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.592520 (+ 3591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.593878 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.594027 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:06.594358 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:06.594441 (+ 83us) server_negotiation.cc:299] Negotiation successful
0620 23:04:06.594642 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6522,"thread_start_us":6379,"threads_started":1}
I20260620 23:04:06.595562 22979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bbe19c8ecf5406f874f46af7ddcc5ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:06.595657 23110 tablet_service.cc:1467] Processing CreateTablet for tablet 2bbe19c8ecf5406f874f46af7ddcc5ac (DEFAULT_TABLE table=test-table [id=8c298bb832ee47e7a3e682cb3da8425d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:04:06.597074 23110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bbe19c8ecf5406f874f46af7ddcc5ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:06.599080 23240 tablet_service.cc:1467] Processing CreateTablet for tablet 2bbe19c8ecf5406f874f46af7ddcc5ac (DEFAULT_TABLE table=test-table [id=8c298bb832ee47e7a3e682cb3da8425d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260620 23:04:06.600190 23240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bbe19c8ecf5406f874f46af7ddcc5ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:06.610466 23323 tablet_bootstrap.cc:492] T 2bbe19c8ecf5406f874f46af7ddcc5ac P ecbfea6397c34614a014989c88eff37c: Bootstrap starting.
I20260620 23:04:06.614362 23325 tablet_bootstrap.cc:492] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873: Bootstrap starting.
I20260620 23:04:06.614638 23044 heartbeater.cc:503] Master 127.2.237.62:34489 was elected leader, sending a full tablet report...
I20260620 23:04:06.615113 23323 tablet_bootstrap.cc:654] T 2bbe19c8ecf5406f874f46af7ddcc5ac P ecbfea6397c34614a014989c88eff37c: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:06.616556 23323 log.cc:824] T 2bbe19c8ecf5406f874f46af7ddcc5ac P ecbfea6397c34614a014989c88eff37c: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:06.620648 23325 tablet_bootstrap.cc:654] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:06.622159 23325 log.cc:824] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:06.623021 23324 tablet_bootstrap.cc:492] T 2bbe19c8ecf5406f874f46af7ddcc5ac P d64af67b60864d03871f3d2c92868cb9: Bootstrap starting.
I20260620 23:04:06.624215 23323 tablet_bootstrap.cc:492] T 2bbe19c8ecf5406f874f46af7ddcc5ac P ecbfea6397c34614a014989c88eff37c: No bootstrap required, opened a new log
I20260620 23:04:06.624539 23323 ts_tablet_manager.cc:1397] T 2bbe19c8ecf5406f874f46af7ddcc5ac P ecbfea6397c34614a014989c88eff37c: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.004s
I20260620 23:04:06.626401 23325 tablet_bootstrap.cc:492] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873: No bootstrap required, opened a new log
I20260620 23:04:06.626729 23325 ts_tablet_manager.cc:1397] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20260620 23:04:06.626878 23324 tablet_bootstrap.cc:654] T 2bbe19c8ecf5406f874f46af7ddcc5ac P d64af67b60864d03871f3d2c92868cb9: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:06.628180 23324 log.cc:824] T 2bbe19c8ecf5406f874f46af7ddcc5ac P d64af67b60864d03871f3d2c92868cb9: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:06.631434 23324 tablet_bootstrap.cc:492] T 2bbe19c8ecf5406f874f46af7ddcc5ac P d64af67b60864d03871f3d2c92868cb9: No bootstrap required, opened a new log
I20260620 23:04:06.631708 23324 ts_tablet_manager.cc:1397] T 2bbe19c8ecf5406f874f46af7ddcc5ac P d64af67b60864d03871f3d2c92868cb9: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260620 23:04:06.633154 23323 raft_consensus.cc:348] T 2bbe19c8ecf5406f874f46af7ddcc5ac P ecbfea6397c34614a014989c88eff37c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d64af67b60864d03871f3d2c92868cb9" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33629 } } peers { permanent_uuid: "9bb35e7fdbaa4b68846341fe9d06c873" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 32947 } } peers { permanent_uuid: "ecbfea6397c34614a014989c88eff37c" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 38465 } }
I20260620 23:04:06.633534 23323 raft_consensus.cc:374] T 2bbe19c8ecf5406f874f46af7ddcc5ac P ecbfea6397c34614a014989c88eff37c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:06.633636 23323 raft_consensus.cc:729] T 2bbe19c8ecf5406f874f46af7ddcc5ac P ecbfea6397c34614a014989c88eff37c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecbfea6397c34614a014989c88eff37c, State: Initialized, Role: FOLLOWER
I20260620 23:04:06.634281 23323 consensus_queue.cc:260] T 2bbe19c8ecf5406f874f46af7ddcc5ac P ecbfea6397c34614a014989c88eff37c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d64af67b60864d03871f3d2c92868cb9" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33629 } } peers { permanent_uuid: "9bb35e7fdbaa4b68846341fe9d06c873" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 32947 } } peers { permanent_uuid: "ecbfea6397c34614a014989c88eff37c" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 38465 } }
I20260620 23:04:06.637331 23305 heartbeater.cc:503] Master 127.2.237.62:34489 was elected leader, sending a full tablet report...
I20260620 23:04:06.637343 23325 raft_consensus.cc:348] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d64af67b60864d03871f3d2c92868cb9" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33629 } } peers { permanent_uuid: "9bb35e7fdbaa4b68846341fe9d06c873" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 32947 } } peers { permanent_uuid: "ecbfea6397c34614a014989c88eff37c" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 38465 } }
I20260620 23:04:06.637771 23325 raft_consensus.cc:374] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:06.637868 23325 raft_consensus.cc:729] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bb35e7fdbaa4b68846341fe9d06c873, State: Initialized, Role: FOLLOWER
I20260620 23:04:06.638074 23323 ts_tablet_manager.cc:1428] T 2bbe19c8ecf5406f874f46af7ddcc5ac P ecbfea6397c34614a014989c88eff37c: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
I20260620 23:04:06.638585 23325 consensus_queue.cc:260] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d64af67b60864d03871f3d2c92868cb9" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33629 } } peers { permanent_uuid: "9bb35e7fdbaa4b68846341fe9d06c873" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 32947 } } peers { permanent_uuid: "ecbfea6397c34614a014989c88eff37c" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 38465 } }
I20260620 23:04:06.643465 23324 raft_consensus.cc:348] T 2bbe19c8ecf5406f874f46af7ddcc5ac P d64af67b60864d03871f3d2c92868cb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d64af67b60864d03871f3d2c92868cb9" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33629 } } peers { permanent_uuid: "9bb35e7fdbaa4b68846341fe9d06c873" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 32947 } } peers { permanent_uuid: "ecbfea6397c34614a014989c88eff37c" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 38465 } }
I20260620 23:04:06.644037 23324 raft_consensus.cc:374] T 2bbe19c8ecf5406f874f46af7ddcc5ac P d64af67b60864d03871f3d2c92868cb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:06.644158 23324 raft_consensus.cc:729] T 2bbe19c8ecf5406f874f46af7ddcc5ac P d64af67b60864d03871f3d2c92868cb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d64af67b60864d03871f3d2c92868cb9, State: Initialized, Role: FOLLOWER
I20260620 23:04:06.644907 23324 consensus_queue.cc:260] T 2bbe19c8ecf5406f874f46af7ddcc5ac P d64af67b60864d03871f3d2c92868cb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d64af67b60864d03871f3d2c92868cb9" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33629 } } peers { permanent_uuid: "9bb35e7fdbaa4b68846341fe9d06c873" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 32947 } } peers { permanent_uuid: "ecbfea6397c34614a014989c88eff37c" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 38465 } }
I20260620 23:04:06.645830 23175 heartbeater.cc:503] Master 127.2.237.62:34489 was elected leader, sending a full tablet report...
I20260620 23:04:06.646852 23325 ts_tablet_manager.cc:1428] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873: Time spent starting tablet: real 0.020s user 0.019s sys 0.000s
I20260620 23:04:06.646843 23324 ts_tablet_manager.cc:1428] T 2bbe19c8ecf5406f874f46af7ddcc5ac P d64af67b60864d03871f3d2c92868cb9: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260620 23:04:06.746140 23330 raft_consensus.cc:482] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260620 23:04:06.746459 23330 raft_consensus.cc:504] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d64af67b60864d03871f3d2c92868cb9" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33629 } } peers { permanent_uuid: "9bb35e7fdbaa4b68846341fe9d06c873" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 32947 } } peers { permanent_uuid: "ecbfea6397c34614a014989c88eff37c" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 38465 } }
I20260620 23:04:06.748215 23330 leader_election.cc:288] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d64af67b60864d03871f3d2c92868cb9 (127.2.237.1:33629), ecbfea6397c34614a014989c88eff37c (127.2.237.3:38465)
I20260620 23:04:06.754873 23319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:06.748828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:36111 (local address 127.2.237.1:33629)
0620 23:04:06.749106 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:06.749112 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:06.749472 (+ 360us) server_negotiation.cc:407] Connection header received
0620 23:04:06.749745 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:06.749755 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:06.749939 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:06.750058 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:04:06.750636 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.751365 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.753163 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.753923 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.754075 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:06.754179 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:06.754455 (+ 276us) server_negotiation.cc:299] Negotiation successful
0620 23:04:06.754608 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260620 23:04:06.754927 23332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:06.748682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.1:33629 (local address 127.2.237.2:36111)
0620 23:04:06.749339 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:06.749356 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:06.749539 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:06.750090 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:06.750095 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:06.750121 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:04:06.750489 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:06.750498 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.751495 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.751503 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:06.752969 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:06.752981 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.754111 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.754117 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:06.754293 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:06.754317 (+ 24us) client_negotiation.cc:769] Sending connection context
0620 23:04:06.754427 (+ 110us) client_negotiation.cc:240] Negotiation successful
0620 23:04:06.754554 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":383,"threads_started":1}
I20260620 23:04:06.755224 23322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:06.749793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.237.2:59859 (local address 127.2.237.3:38465)
0620 23:04:06.749999 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:06.750005 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:06.750200 (+ 195us) server_negotiation.cc:407] Connection header received
0620 23:04:06.750417 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:06.750421 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:06.750481 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:06.750578 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0620 23:04:06.751107 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.751801 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.753544 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.754323 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.754460 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:06.754599 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:06.754864 (+ 265us) server_negotiation.cc:299] Negotiation successful
0620 23:04:06.754987 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260620 23:04:06.755322 23333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:06.749480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.3:38465 (local address 127.2.237.2:59859)
0620 23:04:06.750091 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:06.750104 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:06.750267 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:06.750622 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:06.750626 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:06.750648 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0620 23:04:06.750952 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:06.750960 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.751941 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.751947 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:06.753345 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:06.753357 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.754514 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.754520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:06.754705 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:06.754726 (+ 21us) client_negotiation.cc:769] Sending connection context
0620 23:04:06.754839 (+ 113us) client_negotiation.cc:240] Negotiation successful
0620 23:04:06.754966 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"mutex_wait_us":2,"thread_start_us":318,"threads_started":1}
I20260620 23:04:06.756088 22999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bbe19c8ecf5406f874f46af7ddcc5ac" candidate_uuid: "9bb35e7fdbaa4b68846341fe9d06c873" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d64af67b60864d03871f3d2c92868cb9" is_pre_election: true
I20260620 23:04:06.756255 23260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bbe19c8ecf5406f874f46af7ddcc5ac" candidate_uuid: "9bb35e7fdbaa4b68846341fe9d06c873" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecbfea6397c34614a014989c88eff37c" is_pre_election: true
I20260620 23:04:06.756657 22999 raft_consensus.cc:2454] T 2bbe19c8ecf5406f874f46af7ddcc5ac P d64af67b60864d03871f3d2c92868cb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bb35e7fdbaa4b68846341fe9d06c873 in term 0.
I20260620 23:04:06.756786 23260 raft_consensus.cc:2454] T 2bbe19c8ecf5406f874f46af7ddcc5ac P ecbfea6397c34614a014989c88eff37c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bb35e7fdbaa4b68846341fe9d06c873 in term 0.
I20260620 23:04:06.757341 23066 leader_election.cc:302] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bb35e7fdbaa4b68846341fe9d06c873, d64af67b60864d03871f3d2c92868cb9; no voters:
I20260620 23:04:06.757733 23330 raft_consensus.cc:2781] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260620 23:04:06.757898 23330 raft_consensus.cc:482] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260620 23:04:06.758008 23330 raft_consensus.cc:3037] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:06.760510 23330 raft_consensus.cc:504] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d64af67b60864d03871f3d2c92868cb9" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33629 } } peers { permanent_uuid: "9bb35e7fdbaa4b68846341fe9d06c873" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 32947 } } peers { permanent_uuid: "ecbfea6397c34614a014989c88eff37c" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 38465 } }
I20260620 23:04:06.761266 23330 leader_election.cc:288] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873 [CANDIDATE]: Term 1 election: Requested vote from peers d64af67b60864d03871f3d2c92868cb9 (127.2.237.1:33629), ecbfea6397c34614a014989c88eff37c (127.2.237.3:38465)
I20260620 23:04:06.761773 22999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bbe19c8ecf5406f874f46af7ddcc5ac" candidate_uuid: "9bb35e7fdbaa4b68846341fe9d06c873" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d64af67b60864d03871f3d2c92868cb9"
I20260620 23:04:06.761864 23260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bbe19c8ecf5406f874f46af7ddcc5ac" candidate_uuid: "9bb35e7fdbaa4b68846341fe9d06c873" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecbfea6397c34614a014989c88eff37c"
I20260620 23:04:06.761992 22999 raft_consensus.cc:3037] T 2bbe19c8ecf5406f874f46af7ddcc5ac P d64af67b60864d03871f3d2c92868cb9 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:06.762061 23260 raft_consensus.cc:3037] T 2bbe19c8ecf5406f874f46af7ddcc5ac P ecbfea6397c34614a014989c88eff37c [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:06.765403 23260 raft_consensus.cc:2454] T 2bbe19c8ecf5406f874f46af7ddcc5ac P ecbfea6397c34614a014989c88eff37c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bb35e7fdbaa4b68846341fe9d06c873 in term 1.
I20260620 23:04:06.765403 22999 raft_consensus.cc:2454] T 2bbe19c8ecf5406f874f46af7ddcc5ac P d64af67b60864d03871f3d2c92868cb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bb35e7fdbaa4b68846341fe9d06c873 in term 1.
I20260620 23:04:06.765844 23066 leader_election.cc:302] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bb35e7fdbaa4b68846341fe9d06c873, d64af67b60864d03871f3d2c92868cb9; no voters:
I20260620 23:04:06.766180 23330 raft_consensus.cc:2781] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:06.766709 23330 raft_consensus.cc:686] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873 [term 1 LEADER]: Becoming Leader. State: Replica: 9bb35e7fdbaa4b68846341fe9d06c873, State: Running, Role: LEADER
I20260620 23:04:06.767256 23330 consensus_queue.cc:237] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d64af67b60864d03871f3d2c92868cb9" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33629 } } peers { permanent_uuid: "9bb35e7fdbaa4b68846341fe9d06c873" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 32947 } } peers { permanent_uuid: "ecbfea6397c34614a014989c88eff37c" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 38465 } }
W20260620 23:04:06.770262 23306 tablet.cc:2369] T 2bbe19c8ecf5406f874f46af7ddcc5ac P ecbfea6397c34614a014989c88eff37c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:04:06.772298 22858 catalog_manager.cc:5495] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9bb35e7fdbaa4b68846341fe9d06c873 (127.2.237.2). New cstate: current_term: 1 leader_uuid: "9bb35e7fdbaa4b68846341fe9d06c873" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d64af67b60864d03871f3d2c92868cb9" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bb35e7fdbaa4b68846341fe9d06c873" member_type: VOTER last_known_addr { host: "127.2.237.2" port: 32947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ecbfea6397c34614a014989c88eff37c" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 38465 } health_report { overall_health: UNKNOWN } } }
W20260620 23:04:06.814868 23176 tablet.cc:2369] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:04:06.822134 23320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:06.816727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53328 (local address 127.2.237.2:32947)
0620 23:04:06.816948 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:06.816955 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:06.817133 (+ 178us) server_negotiation.cc:407] Connection header received
0620 23:04:06.817286 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:06.817289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:06.817373 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:06.817500 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:06.817852 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.818608 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.820037 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.820458 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.820550 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:06.821043 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:06.821564 (+ 521us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:06.821678 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:06.821751 (+ 73us) server_negotiation.cc:299] Negotiation successful
0620 23:04:06.821818 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260620 23:04:06.830542 22999 raft_consensus.cc:1261] T 2bbe19c8ecf5406f874f46af7ddcc5ac P d64af67b60864d03871f3d2c92868cb9 [term 1 FOLLOWER]: Refusing update from remote peer 9bb35e7fdbaa4b68846341fe9d06c873: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:04:06.830626 23260 raft_consensus.cc:1261] T 2bbe19c8ecf5406f874f46af7ddcc5ac P ecbfea6397c34614a014989c88eff37c [term 1 FOLLOWER]: Refusing update from remote peer 9bb35e7fdbaa4b68846341fe9d06c873: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:04:06.831728 23334 consensus_queue.cc:1035] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d64af67b60864d03871f3d2c92868cb9" member_type: VOTER last_known_addr { host: "127.2.237.1" port: 33629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:04:06.832120 23330 consensus_queue.cc:1035] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecbfea6397c34614a014989c88eff37c" member_type: VOTER last_known_addr { host: "127.2.237.3" port: 38465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260620 23:04:06.840286 23045 tablet.cc:2369] T 2bbe19c8ecf5406f874f46af7ddcc5ac P d64af67b60864d03871f3d2c92868cb9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:04:06.844175 23337 mvcc.cc:205] Tried to move back new op lower bound from 7299058265407684608 to 7299058265162686464. Current Snapshot: MvccSnapshot[applied={T|T < 7299058265407684608}]
I20260620 23:04:06.859849 22857 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:04:06.860157 22857 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260620 23:04:06.864081 22857 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 22548efa420646d9ae0137a042eb935a: Sending DeleteTablet for 3 replicas of tablet 2bbe19c8ecf5406f874f46af7ddcc5ac
I20260620 23:04:06.865351 23110 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bbe19c8ecf5406f874f46af7ddcc5ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:04:06 UTC) from {username='slave'} at 127.0.0.1:53326
I20260620 23:04:06.865465 22979 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bbe19c8ecf5406f874f46af7ddcc5ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:04:06 UTC) from {username='slave'} at 127.0.0.1:55542
I20260620 23:04:06.866187 23240 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bbe19c8ecf5406f874f46af7ddcc5ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-20 23:04:06 UTC) from {username='slave'} at 127.0.0.1:56024
I20260620 23:04:06.872756 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 22917
I20260620 23:04:06.877220 23347 tablet_replica.cc:330] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873: stopping tablet replica
I20260620 23:04:06.877697 23347 raft_consensus.cc:2229] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873 [term 1 LEADER]: Raft consensus shutting down.
I20260620 23:04:06.878407 23347 raft_consensus.cc:2258] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:04:06.879279 23346 tablet_replica.cc:330] T 2bbe19c8ecf5406f874f46af7ddcc5ac P d64af67b60864d03871f3d2c92868cb9: stopping tablet replica
I20260620 23:04:06.879730 23346 raft_consensus.cc:2229] T 2bbe19c8ecf5406f874f46af7ddcc5ac P d64af67b60864d03871f3d2c92868cb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:04:06.880290 23346 raft_consensus.cc:2258] T 2bbe19c8ecf5406f874f46af7ddcc5ac P d64af67b60864d03871f3d2c92868cb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:04:06.881397 23348 tablet_replica.cc:330] T 2bbe19c8ecf5406f874f46af7ddcc5ac P ecbfea6397c34614a014989c88eff37c: stopping tablet replica
I20260620 23:04:06.881927 23348 raft_consensus.cc:2229] T 2bbe19c8ecf5406f874f46af7ddcc5ac P ecbfea6397c34614a014989c88eff37c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260620 23:04:06.882401 23346 ts_tablet_manager.cc:1905] T 2bbe19c8ecf5406f874f46af7ddcc5ac P d64af67b60864d03871f3d2c92868cb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:04:06.883087 23348 raft_consensus.cc:2258] T 2bbe19c8ecf5406f874f46af7ddcc5ac P ecbfea6397c34614a014989c88eff37c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260620 23:04:06.883354 23347 ts_tablet_manager.cc:1905] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:04:06.889850 23346 ts_tablet_manager.cc:1918] T 2bbe19c8ecf5406f874f46af7ddcc5ac P d64af67b60864d03871f3d2c92868cb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:04:06.890130 23346 log.cc:1196] T 2bbe19c8ecf5406f874f46af7ddcc5ac P d64af67b60864d03871f3d2c92868cb9: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-0/wal/wals/2bbe19c8ecf5406f874f46af7ddcc5ac
I20260620 23:04:06.890753 23346 ts_tablet_manager.cc:1939] T 2bbe19c8ecf5406f874f46af7ddcc5ac P d64af67b60864d03871f3d2c92868cb9: Deleting consensus metadata
I20260620 23:04:06.891371 23347 ts_tablet_manager.cc:1918] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:04:06.891953 23347 log.cc:1196] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-1/wal/wals/2bbe19c8ecf5406f874f46af7ddcc5ac
I20260620 23:04:06.892655 23347 ts_tablet_manager.cc:1939] T 2bbe19c8ecf5406f874f46af7ddcc5ac P 9bb35e7fdbaa4b68846341fe9d06c873: Deleting consensus metadata
I20260620 23:04:06.893006 22845 catalog_manager.cc:4841] TS d64af67b60864d03871f3d2c92868cb9 (127.2.237.1:33629): tablet 2bbe19c8ecf5406f874f46af7ddcc5ac (table test-table [id=8c298bb832ee47e7a3e682cb3da8425d]) successfully deleted
I20260620 23:04:06.894086 23348 ts_tablet_manager.cc:1905] T 2bbe19c8ecf5406f874f46af7ddcc5ac P ecbfea6397c34614a014989c88eff37c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260620 23:04:06.894529 22842 catalog_manager.cc:4841] TS 9bb35e7fdbaa4b68846341fe9d06c873 (127.2.237.2:32947): tablet 2bbe19c8ecf5406f874f46af7ddcc5ac (table test-table [id=8c298bb832ee47e7a3e682cb3da8425d]) successfully deleted
I20260620 23:04:06.895553 23319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:06.874429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55554 (local address 127.2.237.1:33629)
0620 23:04:06.874704 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:06.874714 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:06.886525 (+ 11811us) server_negotiation.cc:407] Connection header received
0620 23:04:06.887036 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:06.887043 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:06.887143 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:06.887221 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:06.887941 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.889132 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.891066 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:06.891715 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:06.891812 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:06.893893 (+ 2081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:06.893903 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:06.893919 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:06.894702 (+ 783us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:06.894763 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:06.894792 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:06.894928 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:06.895079 (+ 151us) server_negotiation.cc:299] Negotiation successful
0620 23:04:06.895191 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260620 23:04:06.896283 23039 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:04:06.901641 23348 ts_tablet_manager.cc:1918] T 2bbe19c8ecf5406f874f46af7ddcc5ac P ecbfea6397c34614a014989c88eff37c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260620 23:04:06.901820 23348 log.cc:1196] T 2bbe19c8ecf5406f874f46af7ddcc5ac P ecbfea6397c34614a014989c88eff37c: Deleting WAL directory at /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781996318807807-2996-0/minicluster-data/ts-2/wal/wals/2bbe19c8ecf5406f874f46af7ddcc5ac
I20260620 23:04:06.902355 23348 ts_tablet_manager.cc:1939] T 2bbe19c8ecf5406f874f46af7ddcc5ac P ecbfea6397c34614a014989c88eff37c: Deleting consensus metadata
I20260620 23:04:06.904331 22845 catalog_manager.cc:4841] TS ecbfea6397c34614a014989c88eff37c (127.2.237.3:38465): tablet 2bbe19c8ecf5406f874f46af7ddcc5ac (table test-table [id=8c298bb832ee47e7a3e682cb3da8425d]) successfully deleted
I20260620 23:04:06.987864 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 22917
I20260620 23:04:07.005904 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 23048
I20260620 23:04:07.013831 23320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:07.006751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53340 (local address 127.2.237.2:32947)
0620 23:04:07.007008 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:07.007015 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:07.007037 (+ 22us) server_negotiation.cc:407] Connection header received
0620 23:04:07.007154 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:07.007160 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:07.007279 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:07.007391 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:07.007867 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:07.009071 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:07.010281 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:07.010993 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:07.011120 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:07.012154 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:07.012164 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:07.012188 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:07.013006 (+ 818us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:07.013068 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:07.013100 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:07.013214 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:07.013338 (+ 124us) server_negotiation.cc:299] Negotiation successful
0620 23:04:07.013439 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260620 23:04:07.014797 23170 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:04:07.128775 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 23048
I20260620 23:04:07.147527 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 23178
I20260620 23:04:07.153988 23322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:07.148284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56034 (local address 127.2.237.3:38465)
0620 23:04:07.148470 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:07.148474 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:07.148535 (+ 61us) server_negotiation.cc:407] Connection header received
0620 23:04:07.148683 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:07.148687 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:07.148765 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:07.148854 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:07.149212 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:07.149966 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:07.151332 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:07.151698 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:07.151800 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:07.152808 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:07.152816 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:07.152830 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:07.153408 (+ 578us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:07.153455 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:07.153473 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:07.153562 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:07.153668 (+ 106us) server_negotiation.cc:299] Negotiation successful
0620 23:04:07.153751 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260620 23:04:07.154713 23300 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:04:07.243467 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 23178
I20260620 23:04:07.260581 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 22827
I20260620 23:04:07.261966 22888 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:04:07.370779 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 22827
2026-06-20T23:04:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2855 ms)
[----------] 6 tests from EncryptionPolicyTest (21720 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:04:07 dist-test-slave-f63b krb5kdc[23356](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:04:07 dist-test-slave-f63b krb5kdc[23356](info): set up 2 sockets
Jun 20 23:04:07 dist-test-slave-f63b krb5kdc[23356](info): commencing operation
krb5kdc: starting...
W20260620 23:04:08.483287 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.002s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 23:04:08 dist-test-slave-f63b krb5kdc[23356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:04:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:04:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:04:08.748546 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42463
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42463
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:09.077036 23369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:09.077371 23369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:09.077538 23369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:09.084829 23369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:09.084991 23369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:09.085089 23369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:09.085191 23369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:09.085265 23369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:09.095753 23369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42463
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:09.097106 23369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:09.098410 23369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:09.110116 23375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:09.110157 23377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:09.110193 23374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:09.111224 23369 server_base.cc:1034] running on GCE node
I20260620 23:04:09.111984 23369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:09.113370 23369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:09.114593 23369 hybrid_clock.cc:648] HybridClock initialized: now 1781996649114524 us; error 33 us; skew 500 ppm
I20260620 23:04:09.114930 23369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:04:09 dist-test-slave-f63b krb5kdc[23356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996649, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:09.117889 23369 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:04:09.119460 23369 webserver.cc:460] Webserver started at http://127.0.0.1:37319/ using document root <none> and password file <none>
I20260620 23:04:09.120160 23369 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:09.120296 23369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:09.120592 23369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:09.123697 23369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "1c3aef24c2e745a59ed0474a0ad90a85"
format_stamp: "Formatted at 2026-06-20 23:04:09 on dist-test-slave-f63b"
server_key: "8c08bbb9d3de6821da065dd016d3f288"
server_key_iv: "4a34dc3ef84d8097086484992beede3e"
server_key_version: "encryptionkey@0"
I20260620 23:04:09.124475 23369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "1c3aef24c2e745a59ed0474a0ad90a85"
format_stamp: "Formatted at 2026-06-20 23:04:09 on dist-test-slave-f63b"
server_key: "8c08bbb9d3de6821da065dd016d3f288"
server_key_iv: "4a34dc3ef84d8097086484992beede3e"
server_key_version: "encryptionkey@0"
I20260620 23:04:09.129599 23369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260620 23:04:09.133138 23384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:09.134390 23369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 23:04:09.134605 23369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "1c3aef24c2e745a59ed0474a0ad90a85"
format_stamp: "Formatted at 2026-06-20 23:04:09 on dist-test-slave-f63b"
server_key: "8c08bbb9d3de6821da065dd016d3f288"
server_key_iv: "4a34dc3ef84d8097086484992beede3e"
server_key_version: "encryptionkey@0"
I20260620 23:04:09.134809 23369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:09.153218 23369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:09.157461 23369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:09.157756 23369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:09.176446 23369 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42463
I20260620 23:04:09.176470 23435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42463 every 8 connection(s)
I20260620 23:04:09.177821 23369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:04:09.181583 23436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:09.182736 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 23369
I20260620 23:04:09.182960 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:04:09.183610 2996 external_mini_cluster.cc:1428] Setting key a6229193f9f4420bf02c77fa3cf9d8a2
I20260620 23:04:09.195528 23436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c3aef24c2e745a59ed0474a0ad90a85: Bootstrap starting.
Jun 20 23:04:09 dist-test-slave-f63b krb5kdc[23356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 23:04:09.199378 23436 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c3aef24c2e745a59ed0474a0ad90a85: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:09.200481 23436 log.cc:824] T 00000000000000000000000000000000 P 1c3aef24c2e745a59ed0474a0ad90a85: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:09.203482 23436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c3aef24c2e745a59ed0474a0ad90a85: No bootstrap required, opened a new log
I20260620 23:04:09.207944 23439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:09.185556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41591 (local address 127.0.0.1:42463)
0620 23:04:09.186387 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:09.186399 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:09.186442 (+ 43us) server_negotiation.cc:407] Connection header received
0620 23:04:09.188094 (+ 1652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:09.188117 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:09.188534 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:09.188868 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:09.190718 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:09.192067 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:09.193913 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:09.194619 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:09.194771 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:09.199709 (+ 4938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:09.199718 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:09.199738 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:09.199794 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:09.203161 (+ 3367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:09.204157 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:09.204163 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:09.204170 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:09.204358 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:09.204928 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:09.204932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:09.204935 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:09.205537 (+ 602us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:09.205924 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:09.206462 (+ 538us) server_negotiation.cc:299] Negotiation successful
0620 23:04:09.206826 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":393,"threads_started":1}
I20260620 23:04:09.212251 2996 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:04:09.212106 23436 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c3aef24c2e745a59ed0474a0ad90a85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c3aef24c2e745a59ed0474a0ad90a85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42463 } }
I20260620 23:04:09.212502 23436 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c3aef24c2e745a59ed0474a0ad90a85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:09.212610 23436 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c3aef24c2e745a59ed0474a0ad90a85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c3aef24c2e745a59ed0474a0ad90a85, State: Initialized, Role: FOLLOWER
I20260620 23:04:09.213197 23436 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c3aef24c2e745a59ed0474a0ad90a85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c3aef24c2e745a59ed0474a0ad90a85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42463 } }
I20260620 23:04:09.213440 23436 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c3aef24c2e745a59ed0474a0ad90a85 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:04:09.213574 23436 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c3aef24c2e745a59ed0474a0ad90a85 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:04:09.213747 23436 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c3aef24c2e745a59ed0474a0ad90a85 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:09.215963 23436 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c3aef24c2e745a59ed0474a0ad90a85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c3aef24c2e745a59ed0474a0ad90a85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42463 } }
I20260620 23:04:09.216436 23436 leader_election.cc:302] T 00000000000000000000000000000000 P 1c3aef24c2e745a59ed0474a0ad90a85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c3aef24c2e745a59ed0474a0ad90a85; no voters:
I20260620 23:04:09.217017 23436 leader_election.cc:288] T 00000000000000000000000000000000 P 1c3aef24c2e745a59ed0474a0ad90a85 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:04:09.217233 23441 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c3aef24c2e745a59ed0474a0ad90a85 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:09.218976 23436 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c3aef24c2e745a59ed0474a0ad90a85 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:09.221127 23441 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c3aef24c2e745a59ed0474a0ad90a85 [term 1 LEADER]: Becoming Leader. State: Replica: 1c3aef24c2e745a59ed0474a0ad90a85, State: Running, Role: LEADER
I20260620 23:04:09.221669 23441 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c3aef24c2e745a59ed0474a0ad90a85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c3aef24c2e745a59ed0474a0ad90a85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42463 } }
I20260620 23:04:09.224217 23443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c3aef24c2e745a59ed0474a0ad90a85 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c3aef24c2e745a59ed0474a0ad90a85. Latest consensus state: current_term: 1 leader_uuid: "1c3aef24c2e745a59ed0474a0ad90a85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c3aef24c2e745a59ed0474a0ad90a85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42463 } } }
I20260620 23:04:09.224829 23442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c3aef24c2e745a59ed0474a0ad90a85 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c3aef24c2e745a59ed0474a0ad90a85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c3aef24c2e745a59ed0474a0ad90a85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42463 } } }
I20260620 23:04:09.225015 23443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c3aef24c2e745a59ed0474a0ad90a85 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:09.225080 23442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c3aef24c2e745a59ed0474a0ad90a85 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:09.239593 23452 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:04:09.245254 23452 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:04:09.255509 23452 catalog_manager.cc:1347] Generated new cluster ID: 6eb02dfaebc04b5493c2a5004fc3abf8
I20260620 23:04:09.255612 23452 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:04:09.256889 23439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:09.233568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51271 (local address 127.0.0.1:42463)
0620 23:04:09.233772 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:09.233781 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:09.239216 (+ 5435us) server_negotiation.cc:407] Connection header received
0620 23:04:09.239549 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:09.239558 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:09.239668 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:09.239825 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:09.241628 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:09.243212 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:09.245084 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:09.245887 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:09.246057 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:09.250032 (+ 3975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:09.250037 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:09.250040 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:09.250070 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:09.253439 (+ 3369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:09.254431 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:09.254436 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:09.254439 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:09.254630 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:09.255216 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:09.255220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:09.255226 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:09.255658 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:09.255934 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:09.256345 (+ 411us) server_negotiation.cc:299] Negotiation successful
0620 23:04:09.256520 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260620 23:04:09.263969 23452 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:04:09.265216 23452 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:04:09.271966 23452 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c3aef24c2e745a59ed0474a0ad90a85: Generated new TSK 0
I20260620 23:04:09.272746 23452 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:04:09.286667 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 23369
I20260620 23:04:09.288467 23431 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:04:09.378903 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 23369
W20260620 23:04:09.396212 23354 connection.cc:535] client connection to 127.0.0.1:42463 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42463): Connection reset by peer (error 104)
2026-06-20T23:04:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2002 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:04:09 dist-test-slave-f63b krb5kdc[23468](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:04:09 dist-test-slave-f63b krb5kdc[23468](info): set up 2 sockets
Jun 20 23:04:09 dist-test-slave-f63b krb5kdc[23468](info): commencing operation
krb5kdc: starting...
W20260620 23:04:10.486950 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 23:04:10 dist-test-slave-f63b krb5kdc[23468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:04:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:04:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:04:10.748070 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40063
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40063
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:11.076974 23481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:11.077250 23481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:11.077445 23481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:11.084785 23481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:11.084899 23481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:11.085003 23481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:11.085093 23481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:11.085150 23481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:11.095083 23481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40063
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:11.096477 23481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:11.097827 23481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:11.109642 23486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:11.109689 23489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:11.109714 23487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:11.110667 23481 server_base.cc:1034] running on GCE node
I20260620 23:04:11.111536 23481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:11.112967 23481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:11.114174 23481 hybrid_clock.cc:648] HybridClock initialized: now 1781996651114117 us; error 49 us; skew 500 ppm
I20260620 23:04:11.114609 23481 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:04:11 dist-test-slave-f63b krb5kdc[23468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996651, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:11.118023 23481 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:04:11.119697 23481 webserver.cc:460] Webserver started at http://127.0.0.1:44673/ using document root <none> and password file <none>
I20260620 23:04:11.120432 23481 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:11.120566 23481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:11.120872 23481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:11.124071 23481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "189eb05a22eb4d978595833c40bf0de1"
format_stamp: "Formatted at 2026-06-20 23:04:11 on dist-test-slave-f63b"
server_key: "7ef94961f19baed26a8e48893fd23e5f"
server_key_iv: "c3ab7391eabe5c9cbdf171c02a7d2cf5"
server_key_version: "encryptionkey@0"
I20260620 23:04:11.124857 23481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "189eb05a22eb4d978595833c40bf0de1"
format_stamp: "Formatted at 2026-06-20 23:04:11 on dist-test-slave-f63b"
server_key: "7ef94961f19baed26a8e48893fd23e5f"
server_key_iv: "c3ab7391eabe5c9cbdf171c02a7d2cf5"
server_key_version: "encryptionkey@0"
I20260620 23:04:11.130120 23481 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260620 23:04:11.133860 23496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:11.135380 23481 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260620 23:04:11.135564 23481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "189eb05a22eb4d978595833c40bf0de1"
format_stamp: "Formatted at 2026-06-20 23:04:11 on dist-test-slave-f63b"
server_key: "7ef94961f19baed26a8e48893fd23e5f"
server_key_iv: "c3ab7391eabe5c9cbdf171c02a7d2cf5"
server_key_version: "encryptionkey@0"
I20260620 23:04:11.135768 23481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:11.161041 23481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:11.165623 23481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:11.165963 23481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:11.185230 23481 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40063
I20260620 23:04:11.185271 23547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40063 every 8 connection(s)
I20260620 23:04:11.186687 23481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:04:11.190696 23548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:11.193140 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 23481
I20260620 23:04:11.193352 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:04:11.193782 2996 external_mini_cluster.cc:1428] Setting key 54d3634bdbb184f840a462a315f81475
I20260620 23:04:11.202661 23548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 189eb05a22eb4d978595833c40bf0de1: Bootstrap starting.
I20260620 23:04:11.206300 23548 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 189eb05a22eb4d978595833c40bf0de1: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:11.207357 23551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:11.195618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35867 (local address 127.0.0.1:40063)
0620 23:04:11.196443 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:11.196455 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:11.196497 (+ 42us) server_negotiation.cc:407] Connection header received
0620 23:04:11.198135 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:11.198157 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:11.198550 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:11.198977 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:11.199570 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:11.200932 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:11.202835 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:11.203496 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:11.203631 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:11.204983 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:11.204989 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:11.205003 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:11.205528 (+ 525us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:11.205602 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:11.205626 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:11.205739 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:11.206012 (+ 273us) server_negotiation.cc:299] Negotiation successful
0620 23:04:11.206221 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":395,"threads_started":1}
I20260620 23:04:11.207437 23548 log.cc:824] T 00000000000000000000000000000000 P 189eb05a22eb4d978595833c40bf0de1: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:11.210753 23548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 189eb05a22eb4d978595833c40bf0de1: No bootstrap required, opened a new log
I20260620 23:04:11.211331 2996 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:04:11.219203 23548 raft_consensus.cc:348] T 00000000000000000000000000000000 P 189eb05a22eb4d978595833c40bf0de1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "189eb05a22eb4d978595833c40bf0de1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40063 } }
I20260620 23:04:11.219657 23548 raft_consensus.cc:374] T 00000000000000000000000000000000 P 189eb05a22eb4d978595833c40bf0de1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:11.219786 23548 raft_consensus.cc:729] T 00000000000000000000000000000000 P 189eb05a22eb4d978595833c40bf0de1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 189eb05a22eb4d978595833c40bf0de1, State: Initialized, Role: FOLLOWER
I20260620 23:04:11.220458 23548 consensus_queue.cc:260] T 00000000000000000000000000000000 P 189eb05a22eb4d978595833c40bf0de1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "189eb05a22eb4d978595833c40bf0de1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40063 } }
I20260620 23:04:11.220723 23548 raft_consensus.cc:388] T 00000000000000000000000000000000 P 189eb05a22eb4d978595833c40bf0de1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:04:11.220878 23548 raft_consensus.cc:482] T 00000000000000000000000000000000 P 189eb05a22eb4d978595833c40bf0de1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:04:11.221076 23548 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 189eb05a22eb4d978595833c40bf0de1 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:11.222059 23551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:11.215689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40429 (local address 127.0.0.1:40063)
0620 23:04:11.215863 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:11.215871 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:11.216244 (+ 373us) server_negotiation.cc:407] Connection header received
0620 23:04:11.216457 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:11.216462 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:11.216532 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:11.216637 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:11.217112 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:11.217956 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:11.219392 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:11.220003 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:11.220128 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:11.221282 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:11.221288 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:11.221303 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:11.221322 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:11.221389 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:11.221405 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:11.221522 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:11.221718 (+ 196us) server_negotiation.cc:299] Negotiation successful
0620 23:04:11.221777 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 23:04:11.224220 23548 raft_consensus.cc:504] T 00000000000000000000000000000000 P 189eb05a22eb4d978595833c40bf0de1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "189eb05a22eb4d978595833c40bf0de1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40063 } }
I20260620 23:04:11.224730 23548 leader_election.cc:302] T 00000000000000000000000000000000 P 189eb05a22eb4d978595833c40bf0de1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 189eb05a22eb4d978595833c40bf0de1; no voters:
I20260620 23:04:11.225333 23548 leader_election.cc:288] T 00000000000000000000000000000000 P 189eb05a22eb4d978595833c40bf0de1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:04:11.225481 23558 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 189eb05a22eb4d978595833c40bf0de1 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:11.226248 23558 raft_consensus.cc:686] T 00000000000000000000000000000000 P 189eb05a22eb4d978595833c40bf0de1 [term 1 LEADER]: Becoming Leader. State: Replica: 189eb05a22eb4d978595833c40bf0de1, State: Running, Role: LEADER
I20260620 23:04:11.226861 23558 consensus_queue.cc:237] T 00000000000000000000000000000000 P 189eb05a22eb4d978595833c40bf0de1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "189eb05a22eb4d978595833c40bf0de1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40063 } }
I20260620 23:04:11.227669 23548 sys_catalog.cc:564] T 00000000000000000000000000000000 P 189eb05a22eb4d978595833c40bf0de1 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:11.229825 23560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 189eb05a22eb4d978595833c40bf0de1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 189eb05a22eb4d978595833c40bf0de1. Latest consensus state: current_term: 1 leader_uuid: "189eb05a22eb4d978595833c40bf0de1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "189eb05a22eb4d978595833c40bf0de1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40063 } } }
I20260620 23:04:11.230161 23560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 189eb05a22eb4d978595833c40bf0de1 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:11.230334 23559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 189eb05a22eb4d978595833c40bf0de1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "189eb05a22eb4d978595833c40bf0de1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "189eb05a22eb4d978595833c40bf0de1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40063 } } }
I20260620 23:04:11.230754 23559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 189eb05a22eb4d978595833c40bf0de1 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:11.232556 23565 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:04:11.238340 23565 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:04:11.247947 23565 catalog_manager.cc:1347] Generated new cluster ID: ec98b5c8a6344bd68ff8eca559e98048
I20260620 23:04:11.248075 23565 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:04:11.272013 23565 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:04:11.273869 23565 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:04:11.278962 23565 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 189eb05a22eb4d978595833c40bf0de1: Generated new TSK 0
I20260620 23:04:11.279788 23565 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:04:11.286598 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 23481
W20260620 23:04:11.288650 23543 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35867
W20260620 23:04:11.289439 2996 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 23481: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260620 23:04:11.289635 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 23481
W20260620 23:04:11.307158 23466 connection.cc:535] client connection to 127.0.0.1:40063 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40063): Connection reset by peer (error 104)
2026-06-20T23:04:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1910 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:04:11 dist-test-slave-f63b krb5kdc[23579](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:04:11 dist-test-slave-f63b krb5kdc[23579](info): set up 2 sockets
Jun 20 23:04:11 dist-test-slave-f63b krb5kdc[23579](info): commencing operation
krb5kdc: starting...
W20260620 23:04:12.431416 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.007s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 23:04:12 dist-test-slave-f63b krb5kdc[23579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:04:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:04:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:04:12.697717 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38491
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38491
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:13.041042 23592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:13.041359 23592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:13.041533 23592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:13.049109 23592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:13.049257 23592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:13.049346 23592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:13.049453 23592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:13.049528 23592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:13.059746 23592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38491
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:13.061107 23592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:13.062460 23592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:13.074982 23597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:13.074982 23598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:13.075001 23600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:13.075632 23592 server_base.cc:1034] running on GCE node
I20260620 23:04:13.076499 23592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:13.077996 23592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:13.079238 23592 hybrid_clock.cc:648] HybridClock initialized: now 1781996653079184 us; error 48 us; skew 500 ppm
I20260620 23:04:13.079620 23592 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:04:13 dist-test-slave-f63b krb5kdc[23579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996653, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:13.082863 23592 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:04:13.084388 23592 webserver.cc:460] Webserver started at http://127.0.0.1:34025/ using document root <none> and password file <none>
I20260620 23:04:13.085126 23592 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:13.085271 23592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:13.085585 23592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:13.088892 23592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "80543ea615be444c80f1852181432364"
format_stamp: "Formatted at 2026-06-20 23:04:13 on dist-test-slave-f63b"
server_key: "b7f84bdd45923789e3df39587bab358c"
server_key_iv: "8a22c09db3527268132abbc469d9c76a"
server_key_version: "encryptionkey@0"
I20260620 23:04:13.089695 23592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "80543ea615be444c80f1852181432364"
format_stamp: "Formatted at 2026-06-20 23:04:13 on dist-test-slave-f63b"
server_key: "b7f84bdd45923789e3df39587bab358c"
server_key_iv: "8a22c09db3527268132abbc469d9c76a"
server_key_version: "encryptionkey@0"
I20260620 23:04:13.095359 23592 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260620 23:04:13.099197 23607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:13.100728 23592 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260620 23:04:13.100955 23592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "80543ea615be444c80f1852181432364"
format_stamp: "Formatted at 2026-06-20 23:04:13 on dist-test-slave-f63b"
server_key: "b7f84bdd45923789e3df39587bab358c"
server_key_iv: "8a22c09db3527268132abbc469d9c76a"
server_key_version: "encryptionkey@0"
I20260620 23:04:13.101188 23592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:13.107501 23592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:13.112128 23592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:13.112445 23592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:13.131345 23592 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38491
I20260620 23:04:13.131376 23658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38491 every 8 connection(s)
I20260620 23:04:13.132840 23592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:04:13.136687 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 23592
I20260620 23:04:13.136893 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:04:13.136935 23659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:13.137480 2996 external_mini_cluster.cc:1428] Setting key 9dd261f76fb81da3c9f5137251811fa6
I20260620 23:04:13.145349 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:13.139410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56983 (local address 127.0.0.1:38491)
0620 23:04:13.140229 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:13.140241 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:13.140281 (+ 40us) server_negotiation.cc:407] Connection header received
0620 23:04:13.141873 (+ 1592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:13.141896 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:13.142284 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:13.142734 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:13.143766 (+ 1032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:13.143772 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:13.143785 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:13.143798 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:13.143859 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:13.143879 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:13.144118 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:13.144270 (+ 152us) server_negotiation.cc:299] Negotiation successful
0620 23:04:13.144463 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":346,"threads_started":1}
I20260620 23:04:13.149039 2996 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:04:13.150367 23659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80543ea615be444c80f1852181432364: Bootstrap starting.
I20260620 23:04:13.153671 23659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80543ea615be444c80f1852181432364: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:13.155128 23659 log.cc:824] T 00000000000000000000000000000000 P 80543ea615be444c80f1852181432364: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:13.156138 23662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:13.153412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60681 (local address 127.0.0.1:38491)
0620 23:04:13.153611 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:13.153619 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:13.153969 (+ 350us) server_negotiation.cc:407] Connection header received
0620 23:04:13.154124 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:13.154131 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:13.154205 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:13.154327 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:13.155337 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:13.155342 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:13.155352 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:13.155368 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:13.155407 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:13.155423 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:13.155544 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:13.155672 (+ 128us) server_negotiation.cc:299] Negotiation successful
0620 23:04:13.155734 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 23:04:13.158138 23659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80543ea615be444c80f1852181432364: No bootstrap required, opened a new log
I20260620 23:04:13.165534 23659 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80543ea615be444c80f1852181432364 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80543ea615be444c80f1852181432364" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38491 } }
I20260620 23:04:13.165989 23659 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80543ea615be444c80f1852181432364 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:13.166110 23659 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80543ea615be444c80f1852181432364 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80543ea615be444c80f1852181432364, State: Initialized, Role: FOLLOWER
I20260620 23:04:13.166749 23659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80543ea615be444c80f1852181432364 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80543ea615be444c80f1852181432364" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38491 } }
I20260620 23:04:13.166991 23659 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80543ea615be444c80f1852181432364 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:04:13.167137 23659 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80543ea615be444c80f1852181432364 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:04:13.167297 23659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80543ea615be444c80f1852181432364 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:13.169543 23659 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80543ea615be444c80f1852181432364 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80543ea615be444c80f1852181432364" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38491 } }
I20260620 23:04:13.170055 23659 leader_election.cc:302] T 00000000000000000000000000000000 P 80543ea615be444c80f1852181432364 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80543ea615be444c80f1852181432364; no voters:
I20260620 23:04:13.171073 23659 leader_election.cc:288] T 00000000000000000000000000000000 P 80543ea615be444c80f1852181432364 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:04:13.171222 23669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80543ea615be444c80f1852181432364 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:13.171882 23669 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80543ea615be444c80f1852181432364 [term 1 LEADER]: Becoming Leader. State: Replica: 80543ea615be444c80f1852181432364, State: Running, Role: LEADER
I20260620 23:04:13.172461 23669 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80543ea615be444c80f1852181432364 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80543ea615be444c80f1852181432364" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38491 } }
I20260620 23:04:13.173245 23659 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80543ea615be444c80f1852181432364 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:13.175269 23671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80543ea615be444c80f1852181432364 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80543ea615be444c80f1852181432364. Latest consensus state: current_term: 1 leader_uuid: "80543ea615be444c80f1852181432364" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80543ea615be444c80f1852181432364" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38491 } } }
I20260620 23:04:13.175649 23671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80543ea615be444c80f1852181432364 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:13.175072 23670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80543ea615be444c80f1852181432364 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80543ea615be444c80f1852181432364" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80543ea615be444c80f1852181432364" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38491 } } }
I20260620 23:04:13.176774 23670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80543ea615be444c80f1852181432364 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:13.178251 23675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:04:13.183599 23675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:04:13.193018 23675 catalog_manager.cc:1347] Generated new cluster ID: d291a71940f34638abb346b56e2291a0
I20260620 23:04:13.193145 23675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:04:13.223063 23675 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:04:13.223637 23675 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:04:13.231101 23675 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80543ea615be444c80f1852181432364: Generated new TSK 0
I20260620 23:04:13.232021 23675 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:04:13.240276 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 23592
W20260620 23:04:13.241437 23654 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:56983
W20260620 23:04:13.242189 2996 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 23592: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260620 23:04:13.242336 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 23592
2026-06-20T23:04:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1957 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:04:13 dist-test-slave-f63b krb5kdc[23690](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:04:13 dist-test-slave-f63b krb5kdc[23690](info): set up 2 sockets
Jun 20 23:04:13 dist-test-slave-f63b krb5kdc[23690](info): commencing operation
krb5kdc: starting...
W20260620 23:04:14.375501 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 23:04:14 dist-test-slave-f63b krb5kdc[23690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:04:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:04:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:04:14.639249 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40405
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:14.992367 23703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:14.992744 23703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:14.992911 23703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:15.000983 23703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:15.001145 23703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:15.001243 23703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:15.001353 23703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:15.001422 23703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:15.012073 23703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:15.013546 23703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:15.015009 23703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:15.027825 23708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:15.027769 23711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:15.027868 23709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:15.028705 23703 server_base.cc:1034] running on GCE node
I20260620 23:04:15.029517 23703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:15.031065 23703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:15.032315 23703 hybrid_clock.cc:648] HybridClock initialized: now 1781996655032259 us; error 41 us; skew 500 ppm
I20260620 23:04:15.032707 23703 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:04:15 dist-test-slave-f63b krb5kdc[23690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996655, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:15.040777 23703 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:04:15.042659 23703 webserver.cc:460] Webserver started at http://127.0.0.1:36777/ using document root <none> and password file <none>
I20260620 23:04:15.043453 23703 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:15.043603 23703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:15.043922 23703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:15.047097 23703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "6124f14b337e49248f10b6e49b88296c"
format_stamp: "Formatted at 2026-06-20 23:04:15 on dist-test-slave-f63b"
server_key: "4ab89eafd592e42cb427446913012dfe"
server_key_iv: "04cea984b95076cc0906564e36837c67"
server_key_version: "encryptionkey@0"
I20260620 23:04:15.047922 23703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "6124f14b337e49248f10b6e49b88296c"
format_stamp: "Formatted at 2026-06-20 23:04:15 on dist-test-slave-f63b"
server_key: "4ab89eafd592e42cb427446913012dfe"
server_key_iv: "04cea984b95076cc0906564e36837c67"
server_key_version: "encryptionkey@0"
I20260620 23:04:15.053682 23703 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260620 23:04:15.058190 23718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:15.060101 23703 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260620 23:04:15.060359 23703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "6124f14b337e49248f10b6e49b88296c"
format_stamp: "Formatted at 2026-06-20 23:04:15 on dist-test-slave-f63b"
server_key: "4ab89eafd592e42cb427446913012dfe"
server_key_iv: "04cea984b95076cc0906564e36837c67"
server_key_version: "encryptionkey@0"
I20260620 23:04:15.060614 23703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:15.079030 23703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:15.083174 23703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:15.083434 23703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:15.101791 23703 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40405
I20260620 23:04:15.101890 23769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40405 every 8 connection(s)
I20260620 23:04:15.103251 23703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:04:15.105923 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 23703
I20260620 23:04:15.106107 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:04:15.106611 2996 external_mini_cluster.cc:1428] Setting key 6092b485ffb8ce069e0d6e43392b07d4
I20260620 23:04:15.107568 23770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:15.120464 23770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6124f14b337e49248f10b6e49b88296c: Bootstrap starting.
Jun 20 23:04:15 dist-test-slave-f63b krb5kdc[23690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 23:04:15.124006 23770 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6124f14b337e49248f10b6e49b88296c: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:15.125138 23770 log.cc:824] T 00000000000000000000000000000000 P 6124f14b337e49248f10b6e49b88296c: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:15.128165 23770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6124f14b337e49248f10b6e49b88296c: No bootstrap required, opened a new log
I20260620 23:04:15.129963 23773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:15.108469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43421 (local address 127.0.0.1:40405)
0620 23:04:15.109298 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:15.109311 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:15.109357 (+ 46us) server_negotiation.cc:407] Connection header received
0620 23:04:15.111226 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:15.111250 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:15.111724 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:15.112172 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:15.113813 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:15.115303 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:15.116769 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:15.117585 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:15.117866 (+ 281us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:15.122295 (+ 4429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:15.122304 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:15.122320 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:15.122368 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:15.125654 (+ 3286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:15.126501 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:15.126508 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:15.126515 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:15.126746 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:15.127166 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:15.127172 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:15.127176 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:15.127727 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:15.128051 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:15.128488 (+ 437us) server_negotiation.cc:299] Negotiation successful
0620 23:04:15.128959 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":360,"threads_started":1}
I20260620 23:04:15.133723 2996 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:04:15.136227 23770 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6124f14b337e49248f10b6e49b88296c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6124f14b337e49248f10b6e49b88296c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40405 } }
I20260620 23:04:15.136705 23770 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6124f14b337e49248f10b6e49b88296c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:15.136876 23770 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6124f14b337e49248f10b6e49b88296c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6124f14b337e49248f10b6e49b88296c, State: Initialized, Role: FOLLOWER
I20260620 23:04:15.137446 23770 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6124f14b337e49248f10b6e49b88296c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6124f14b337e49248f10b6e49b88296c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40405 } }
I20260620 23:04:15.137681 23770 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6124f14b337e49248f10b6e49b88296c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:04:15.137814 23770 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6124f14b337e49248f10b6e49b88296c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:04:15.138064 23770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6124f14b337e49248f10b6e49b88296c [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:15.140463 23770 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6124f14b337e49248f10b6e49b88296c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6124f14b337e49248f10b6e49b88296c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40405 } }
I20260620 23:04:15.140959 23770 leader_election.cc:302] T 00000000000000000000000000000000 P 6124f14b337e49248f10b6e49b88296c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6124f14b337e49248f10b6e49b88296c; no voters:
I20260620 23:04:15.141678 23770 leader_election.cc:288] T 00000000000000000000000000000000 P 6124f14b337e49248f10b6e49b88296c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:04:15.141816 23780 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6124f14b337e49248f10b6e49b88296c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:15.144450 23770 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6124f14b337e49248f10b6e49b88296c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:15.149845 23780 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6124f14b337e49248f10b6e49b88296c [term 1 LEADER]: Becoming Leader. State: Replica: 6124f14b337e49248f10b6e49b88296c, State: Running, Role: LEADER
I20260620 23:04:15.151501 23780 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6124f14b337e49248f10b6e49b88296c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6124f14b337e49248f10b6e49b88296c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40405 } }
I20260620 23:04:15.154096 23773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:15.138102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38675 (local address 127.0.0.1:40405)
0620 23:04:15.138269 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:15.138273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:15.138979 (+ 706us) server_negotiation.cc:407] Connection header received
0620 23:04:15.139167 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:15.139172 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:15.139246 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:15.139353 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:15.140985 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:15.141817 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:15.143394 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:15.143983 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:15.144109 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:15.146672 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:15.146680 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:15.146687 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:15.146750 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:15.150639 (+ 3889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:15.151464 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:15.151472 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:15.151476 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:15.151744 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:15.152258 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:15.152264 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:15.152267 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:15.152689 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:15.152977 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:15.153244 (+ 267us) server_negotiation.cc:299] Negotiation successful
0620 23:04:15.153547 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260620 23:04:15.155352 23781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6124f14b337e49248f10b6e49b88296c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6124f14b337e49248f10b6e49b88296c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6124f14b337e49248f10b6e49b88296c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40405 } } }
I20260620 23:04:15.155792 23781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6124f14b337e49248f10b6e49b88296c [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:15.156644 23794 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:04:15.156546 23782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6124f14b337e49248f10b6e49b88296c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6124f14b337e49248f10b6e49b88296c. Latest consensus state: current_term: 1 leader_uuid: "6124f14b337e49248f10b6e49b88296c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6124f14b337e49248f10b6e49b88296c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40405 } } }
I20260620 23:04:15.156939 23782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6124f14b337e49248f10b6e49b88296c [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:15.168520 23794 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:04:15.177306 23794 catalog_manager.cc:1347] Generated new cluster ID: d5397a4cb86947a2a0e82029c86f1e7b
I20260620 23:04:15.177431 23794 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:04:15.208061 23794 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:04:15.209389 23794 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:04:15.226447 23794 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6124f14b337e49248f10b6e49b88296c: Generated new TSK 0
I20260620 23:04:15.227562 23794 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:04:15.246044 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 23703
I20260620 23:04:15.247117 23765 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:04:15.344518 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 23703
2026-06-20T23:04:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2096 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:04:15 dist-test-slave-f63b krb5kdc[23802](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:04:15 dist-test-slave-f63b krb5kdc[23802](info): set up 2 sockets
Jun 20 23:04:15 dist-test-slave-f63b krb5kdc[23802](info): commencing operation
krb5kdc: starting...
W20260620 23:04:16.458987 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.003s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 23:04:16 dist-test-slave-f63b krb5kdc[23802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:04:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:04:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:04:16.711726 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45737
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:17.049178 23815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:17.049528 23815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:17.049683 23815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:17.057476 23815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:17.057627 23815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:17.057746 23815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:17.057847 23815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:17.057919 23815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:17.068387 23815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45737
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:17.069751 23815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:17.071159 23815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:17.083032 23820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:17.083113 23821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:17.083109 23823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:17.083925 23815 server_base.cc:1034] running on GCE node
I20260620 23:04:17.084712 23815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:17.086123 23815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:17.087347 23815 hybrid_clock.cc:648] HybridClock initialized: now 1781996657087299 us; error 44 us; skew 500 ppm
I20260620 23:04:17.087704 23815 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:04:17 dist-test-slave-f63b krb5kdc[23802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996657, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:17.090881 23815 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:04:17.092420 23815 webserver.cc:460] Webserver started at http://127.0.0.1:40955/ using document root <none> and password file <none>
I20260620 23:04:17.093124 23815 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:17.093266 23815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:17.093569 23815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:17.096635 23815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "c59b76b924e2470db7d2fe1a0473473e"
format_stamp: "Formatted at 2026-06-20 23:04:17 on dist-test-slave-f63b"
server_key: "71f10eaa57b2d416196b44bc79406141"
server_key_iv: "a3b5ebbc54a94fdf47868a7f3c26d366"
server_key_version: "encryptionkey@0"
I20260620 23:04:17.097433 23815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "c59b76b924e2470db7d2fe1a0473473e"
format_stamp: "Formatted at 2026-06-20 23:04:17 on dist-test-slave-f63b"
server_key: "71f10eaa57b2d416196b44bc79406141"
server_key_iv: "a3b5ebbc54a94fdf47868a7f3c26d366"
server_key_version: "encryptionkey@0"
I20260620 23:04:17.102682 23815 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260620 23:04:17.106129 23830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:17.107595 23815 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260620 23:04:17.107791 23815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "c59b76b924e2470db7d2fe1a0473473e"
format_stamp: "Formatted at 2026-06-20 23:04:17 on dist-test-slave-f63b"
server_key: "71f10eaa57b2d416196b44bc79406141"
server_key_iv: "a3b5ebbc54a94fdf47868a7f3c26d366"
server_key_version: "encryptionkey@0"
I20260620 23:04:17.107985 23815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:17.133219 23815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:17.137965 23815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:17.138266 23815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:17.157331 23815 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45737
I20260620 23:04:17.157354 23881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45737 every 8 connection(s)
I20260620 23:04:17.158773 23815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:04:17.162456 23882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:17.168064 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 23815
I20260620 23:04:17.168260 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:04:17.168674 2996 external_mini_cluster.cc:1428] Setting key 5bdb24807d98fe3c33416e96536a4b6b
I20260620 23:04:17.173146 23882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c59b76b924e2470db7d2fe1a0473473e: Bootstrap starting.
I20260620 23:04:17.176647 23882 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c59b76b924e2470db7d2fe1a0473473e: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:17.177840 23882 log.cc:824] T 00000000000000000000000000000000 P c59b76b924e2470db7d2fe1a0473473e: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:17.181044 23882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c59b76b924e2470db7d2fe1a0473473e: No bootstrap required, opened a new log
I20260620 23:04:17.182024 23885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:17.170400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48021 (local address 127.0.0.1:45737)
0620 23:04:17.171297 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:17.171311 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:17.171352 (+ 41us) server_negotiation.cc:407] Connection header received
0620 23:04:17.172997 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:17.173019 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:17.173448 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:17.173763 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:17.174340 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:17.175695 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:17.177538 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:17.178294 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:17.178520 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:17.179780 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:17.179787 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:17.179800 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:17.180336 (+ 536us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:17.180408 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:17.180435 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:17.180521 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:17.180748 (+ 227us) server_negotiation.cc:299] Negotiation successful
0620 23:04:17.181010 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":459,"threads_started":1}
I20260620 23:04:17.185626 2996 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:04:17.188830 23882 raft_consensus.cc:348] T 00000000000000000000000000000000 P c59b76b924e2470db7d2fe1a0473473e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c59b76b924e2470db7d2fe1a0473473e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45737 } }
I20260620 23:04:17.189191 23882 raft_consensus.cc:374] T 00000000000000000000000000000000 P c59b76b924e2470db7d2fe1a0473473e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:17.189323 23882 raft_consensus.cc:729] T 00000000000000000000000000000000 P c59b76b924e2470db7d2fe1a0473473e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c59b76b924e2470db7d2fe1a0473473e, State: Initialized, Role: FOLLOWER
I20260620 23:04:17.190022 23882 consensus_queue.cc:260] T 00000000000000000000000000000000 P c59b76b924e2470db7d2fe1a0473473e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c59b76b924e2470db7d2fe1a0473473e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45737 } }
I20260620 23:04:17.190346 23882 raft_consensus.cc:388] T 00000000000000000000000000000000 P c59b76b924e2470db7d2fe1a0473473e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:04:17.190493 23882 raft_consensus.cc:482] T 00000000000000000000000000000000 P c59b76b924e2470db7d2fe1a0473473e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:04:17.190712 23882 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c59b76b924e2470db7d2fe1a0473473e [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:17.193766 23882 raft_consensus.cc:504] T 00000000000000000000000000000000 P c59b76b924e2470db7d2fe1a0473473e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c59b76b924e2470db7d2fe1a0473473e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45737 } }
I20260620 23:04:17.194357 23882 leader_election.cc:302] T 00000000000000000000000000000000 P c59b76b924e2470db7d2fe1a0473473e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c59b76b924e2470db7d2fe1a0473473e; no voters:
I20260620 23:04:17.195042 23882 leader_election.cc:288] T 00000000000000000000000000000000 P c59b76b924e2470db7d2fe1a0473473e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:04:17.195227 23892 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c59b76b924e2470db7d2fe1a0473473e [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:17.196229 23892 raft_consensus.cc:686] T 00000000000000000000000000000000 P c59b76b924e2470db7d2fe1a0473473e [term 1 LEADER]: Becoming Leader. State: Replica: c59b76b924e2470db7d2fe1a0473473e, State: Running, Role: LEADER
I20260620 23:04:17.196842 23892 consensus_queue.cc:237] T 00000000000000000000000000000000 P c59b76b924e2470db7d2fe1a0473473e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c59b76b924e2470db7d2fe1a0473473e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45737 } }
I20260620 23:04:17.197479 23885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:17.189907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59189 (local address 127.0.0.1:45737)
0620 23:04:17.190116 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:17.190125 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:17.190605 (+ 480us) server_negotiation.cc:407] Connection header received
0620 23:04:17.190770 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:17.190774 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:17.190869 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:17.190957 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:17.191583 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:17.192325 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:17.194175 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:17.194845 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:17.195001 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:17.196309 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:17.196315 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:17.196323 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:17.196343 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:17.196393 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:17.196413 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:17.196500 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:17.197070 (+ 570us) server_negotiation.cc:299] Negotiation successful
0620 23:04:17.197156 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260620 23:04:17.197561 23882 sys_catalog.cc:564] T 00000000000000000000000000000000 P c59b76b924e2470db7d2fe1a0473473e [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:17.199909 23894 sys_catalog.cc:455] T 00000000000000000000000000000000 P c59b76b924e2470db7d2fe1a0473473e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c59b76b924e2470db7d2fe1a0473473e. Latest consensus state: current_term: 1 leader_uuid: "c59b76b924e2470db7d2fe1a0473473e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c59b76b924e2470db7d2fe1a0473473e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45737 } } }
I20260620 23:04:17.199918 23893 sys_catalog.cc:455] T 00000000000000000000000000000000 P c59b76b924e2470db7d2fe1a0473473e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c59b76b924e2470db7d2fe1a0473473e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c59b76b924e2470db7d2fe1a0473473e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45737 } } }
I20260620 23:04:17.200356 23894 sys_catalog.cc:458] T 00000000000000000000000000000000 P c59b76b924e2470db7d2fe1a0473473e [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:17.200356 23893 sys_catalog.cc:458] T 00000000000000000000000000000000 P c59b76b924e2470db7d2fe1a0473473e [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:17.201308 23897 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:04:17.206319 23897 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:04:17.215451 23897 catalog_manager.cc:1347] Generated new cluster ID: a7e272495f68437eb43033d6b6d9b2ee
I20260620 23:04:17.215565 23897 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:04:17.254861 23897 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:04:17.256163 23897 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:04:17.262990 23897 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c59b76b924e2470db7d2fe1a0473473e: Generated new TSK 0
I20260620 23:04:17.264114 23897 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:04:17.281488 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 23815
W20260620 23:04:17.282745 23877 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48021
W20260620 23:04:17.283447 2996 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 23815: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260620 23:04:17.283576 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 23815
2026-06-20T23:04:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1939 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:04:17 dist-test-slave-f63b krb5kdc[23913](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:04:17 dist-test-slave-f63b krb5kdc[23913](info): set up 2 sockets
Jun 20 23:04:17 dist-test-slave-f63b krb5kdc[23913](info): commencing operation
krb5kdc: starting...
W20260620 23:04:18.399063 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 23:04:18 dist-test-slave-f63b krb5kdc[23913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:04:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:04:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:04:18.657997 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38503
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:19.001384 23926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:19.001762 23926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:19.001950 23926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:19.015653 23926 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:19.015832 23926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:19.015934 23926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:19.016032 23926 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:19.016114 23926 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:19.029664 23926 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:19.031204 23926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:19.032660 23926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:19.045051 23934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:19.045107 23931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:19.045132 23932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:19.046052 23926 server_base.cc:1034] running on GCE node
I20260620 23:04:19.047001 23926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:19.048552 23926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:19.049784 23926 hybrid_clock.cc:648] HybridClock initialized: now 1781996659049745 us; error 42 us; skew 500 ppm
I20260620 23:04:19.050158 23926 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:04:19 dist-test-slave-f63b krb5kdc[23913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996659, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:19.053468 23926 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:04:19.055145 23926 webserver.cc:460] Webserver started at http://127.0.0.1:38987/ using document root <none> and password file <none>
I20260620 23:04:19.055897 23926 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:19.056051 23926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:19.056377 23926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:19.059603 23926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "6b61baa9c69847b799c1439cd8c40c15"
format_stamp: "Formatted at 2026-06-20 23:04:19 on dist-test-slave-f63b"
server_key: "43ba8ca46f693e0114de973cd8eb7203"
server_key_iv: "c9bb5659db5c265a59fed2a7d3ee1bf0"
server_key_version: "encryptionkey@0"
I20260620 23:04:19.060429 23926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "6b61baa9c69847b799c1439cd8c40c15"
format_stamp: "Formatted at 2026-06-20 23:04:19 on dist-test-slave-f63b"
server_key: "43ba8ca46f693e0114de973cd8eb7203"
server_key_iv: "c9bb5659db5c265a59fed2a7d3ee1bf0"
server_key_version: "encryptionkey@0"
I20260620 23:04:19.065984 23926 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260620 23:04:19.069713 23941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:19.071105 23926 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 23:04:19.071300 23926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "6b61baa9c69847b799c1439cd8c40c15"
format_stamp: "Formatted at 2026-06-20 23:04:19 on dist-test-slave-f63b"
server_key: "43ba8ca46f693e0114de973cd8eb7203"
server_key_iv: "c9bb5659db5c265a59fed2a7d3ee1bf0"
server_key_version: "encryptionkey@0"
I20260620 23:04:19.071516 23926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:19.077531 23926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:19.082026 23926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:19.082341 23926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:19.101507 23926 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38503
I20260620 23:04:19.101553 23992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38503 every 8 connection(s)
I20260620 23:04:19.103003 23926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:04:19.104058 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 23926
I20260620 23:04:19.104272 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:04:19.104707 2996 external_mini_cluster.cc:1428] Setting key 6990a68e4543142b3ef4bd16f2c15829
I20260620 23:04:19.107450 23993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:19.112573 23996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:19.106517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33145 (local address 127.0.0.1:38503)
0620 23:04:19.107396 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:19.107420 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:19.107464 (+ 44us) server_negotiation.cc:407] Connection header received
0620 23:04:19.109133 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:19.109154 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:19.109493 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:19.109840 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:19.110967 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:19.110973 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:19.110986 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:19.111000 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:19.111074 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:19.111101 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:19.111272 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:19.111429 (+ 157us) server_negotiation.cc:299] Negotiation successful
0620 23:04:19.111643 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":386,"threads_started":1}
I20260620 23:04:19.116634 2996 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:04:19.120916 23993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b61baa9c69847b799c1439cd8c40c15: Bootstrap starting.
I20260620 23:04:19.124727 23996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:19.122126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59299 (local address 127.0.0.1:38503)
0620 23:04:19.122346 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:19.122352 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:19.122707 (+ 355us) server_negotiation.cc:407] Connection header received
0620 23:04:19.122915 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:19.122919 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:19.122971 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:19.123061 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:19.124022 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:19.124025 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:19.124028 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:19.124043 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:19.124084 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:19.124099 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:19.124192 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:19.124348 (+ 156us) server_negotiation.cc:299] Negotiation successful
0620 23:04:19.124397 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260620 23:04:19.125130 23993 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b61baa9c69847b799c1439cd8c40c15: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:19.126258 23993 log.cc:824] T 00000000000000000000000000000000 P 6b61baa9c69847b799c1439cd8c40c15: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:19.129068 23993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b61baa9c69847b799c1439cd8c40c15: No bootstrap required, opened a new log
I20260620 23:04:19.136863 23993 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b61baa9c69847b799c1439cd8c40c15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b61baa9c69847b799c1439cd8c40c15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38503 } }
I20260620 23:04:19.137297 23993 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b61baa9c69847b799c1439cd8c40c15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:19.137420 23993 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b61baa9c69847b799c1439cd8c40c15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b61baa9c69847b799c1439cd8c40c15, State: Initialized, Role: FOLLOWER
I20260620 23:04:19.138029 23993 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b61baa9c69847b799c1439cd8c40c15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b61baa9c69847b799c1439cd8c40c15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38503 } }
I20260620 23:04:19.138269 23993 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b61baa9c69847b799c1439cd8c40c15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:04:19.138428 23993 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b61baa9c69847b799c1439cd8c40c15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:04:19.138644 23993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b61baa9c69847b799c1439cd8c40c15 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:19.141080 23993 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b61baa9c69847b799c1439cd8c40c15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b61baa9c69847b799c1439cd8c40c15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38503 } }
I20260620 23:04:19.141608 23993 leader_election.cc:302] T 00000000000000000000000000000000 P 6b61baa9c69847b799c1439cd8c40c15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b61baa9c69847b799c1439cd8c40c15; no voters:
I20260620 23:04:19.142171 23993 leader_election.cc:288] T 00000000000000000000000000000000 P 6b61baa9c69847b799c1439cd8c40c15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:04:19.142330 24003 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b61baa9c69847b799c1439cd8c40c15 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:19.143133 24003 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b61baa9c69847b799c1439cd8c40c15 [term 1 LEADER]: Becoming Leader. State: Replica: 6b61baa9c69847b799c1439cd8c40c15, State: Running, Role: LEADER
I20260620 23:04:19.143657 24003 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b61baa9c69847b799c1439cd8c40c15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b61baa9c69847b799c1439cd8c40c15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38503 } }
I20260620 23:04:19.144342 23993 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b61baa9c69847b799c1439cd8c40c15 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:19.146760 24005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b61baa9c69847b799c1439cd8c40c15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b61baa9c69847b799c1439cd8c40c15. Latest consensus state: current_term: 1 leader_uuid: "6b61baa9c69847b799c1439cd8c40c15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b61baa9c69847b799c1439cd8c40c15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38503 } } }
I20260620 23:04:19.147327 24005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b61baa9c69847b799c1439cd8c40c15 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:19.146795 24004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b61baa9c69847b799c1439cd8c40c15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b61baa9c69847b799c1439cd8c40c15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b61baa9c69847b799c1439cd8c40c15" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38503 } } }
I20260620 23:04:19.149598 24004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b61baa9c69847b799c1439cd8c40c15 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:19.154456 24012 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:04:19.158959 24012 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:04:19.167583 24012 catalog_manager.cc:1347] Generated new cluster ID: f6f551adf12942ffb60091e325e22569
I20260620 23:04:19.167719 24012 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:04:19.188884 24012 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:04:19.189440 24012 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:04:19.207160 24012 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b61baa9c69847b799c1439cd8c40c15: Generated new TSK 0
I20260620 23:04:19.208299 24012 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:04:19.225075 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 23926
W20260620 23:04:19.226217 23988 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33145
W20260620 23:04:19.227070 2996 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 23926: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260620 23:04:19.227222 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 23926
2026-06-20T23:04:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1943 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:04:19 dist-test-slave-f63b krb5kdc[24024](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:04:19 dist-test-slave-f63b krb5kdc[24024](info): set up 2 sockets
Jun 20 23:04:19 dist-test-slave-f63b krb5kdc[24024](info): commencing operation
krb5kdc: starting...
W20260620 23:04:20.354923 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 23:04:20 dist-test-slave-f63b krb5kdc[24024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:04:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:04:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:04:20.618232 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36173
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:20.954130 24037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:20.954425 24037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:20.954640 24037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:20.962018 24037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:20.962144 24037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:20.962240 24037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:20.962337 24037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:20.962428 24037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:20.972803 24037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:20.974244 24037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:20.975680 24037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:20.987993 24042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:20.987900 24043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:20.987900 24045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:20.989023 24037 server_base.cc:1034] running on GCE node
I20260620 23:04:20.989832 24037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:20.991276 24037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:20.992482 24037 hybrid_clock.cc:648] HybridClock initialized: now 1781996660992417 us; error 51 us; skew 500 ppm
I20260620 23:04:20.992849 24037 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:04:20 dist-test-slave-f63b krb5kdc[24024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996660, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:21.001174 24037 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:04:21.002831 24037 webserver.cc:460] Webserver started at http://127.0.0.1:35541/ using document root <none> and password file <none>
I20260620 23:04:21.003578 24037 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:21.003729 24037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:21.004056 24037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:21.007340 24037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "7ef7aa220c7549de9a3ab312718bdcf6"
format_stamp: "Formatted at 2026-06-20 23:04:21 on dist-test-slave-f63b"
server_key: "ebcd660f4ce8f27d8971037b3f8ef69d"
server_key_iv: "0e7ca8dc3a64c279bf4a65852e88dec6"
server_key_version: "encryptionkey@0"
I20260620 23:04:21.008428 24037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "7ef7aa220c7549de9a3ab312718bdcf6"
format_stamp: "Formatted at 2026-06-20 23:04:21 on dist-test-slave-f63b"
server_key: "ebcd660f4ce8f27d8971037b3f8ef69d"
server_key_iv: "0e7ca8dc3a64c279bf4a65852e88dec6"
server_key_version: "encryptionkey@0"
I20260620 23:04:21.015754 24037 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260620 23:04:21.019299 24052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:21.020818 24037 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 23:04:21.021015 24037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "7ef7aa220c7549de9a3ab312718bdcf6"
format_stamp: "Formatted at 2026-06-20 23:04:21 on dist-test-slave-f63b"
server_key: "ebcd660f4ce8f27d8971037b3f8ef69d"
server_key_iv: "0e7ca8dc3a64c279bf4a65852e88dec6"
server_key_version: "encryptionkey@0"
I20260620 23:04:21.021221 24037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:21.047886 24037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:21.052270 24037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:21.052567 24037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:21.071652 24037 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36173
I20260620 23:04:21.071676 24103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36173 every 8 connection(s)
I20260620 23:04:21.073102 24037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:04:21.074751 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 24037
I20260620 23:04:21.074934 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:04:21.075381 2996 external_mini_cluster.cc:1428] Setting key c1e74c2566c2d857a35b295115a4dcb7
I20260620 23:04:21.077445 24104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 20 23:04:21 dist-test-slave-f63b krb5kdc[24024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 23:04:21.091333 24104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ef7aa220c7549de9a3ab312718bdcf6: Bootstrap starting.
I20260620 23:04:21.094799 24104 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ef7aa220c7549de9a3ab312718bdcf6: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:21.095829 24104 log.cc:824] T 00000000000000000000000000000000 P 7ef7aa220c7549de9a3ab312718bdcf6: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:21.099344 24104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ef7aa220c7549de9a3ab312718bdcf6: No bootstrap required, opened a new log
I20260620 23:04:21.100021 24107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:21.077223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.217:45093 (local address 127.0.0.1:36173)
0620 23:04:21.078087 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:21.078099 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:21.078139 (+ 40us) server_negotiation.cc:407] Connection header received
0620 23:04:21.079921 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:21.079943 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:21.080366 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:21.080697 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:21.082499 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:21.083953 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:21.085760 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:21.086439 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:21.086650 (+ 211us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:21.091593 (+ 4943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:21.091602 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:21.091617 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:21.091655 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:21.095264 (+ 3609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:21.096111 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:21.096119 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:21.096127 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:21.096350 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:21.096906 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:21.096909 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:21.096912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:21.097599 (+ 687us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:21.097924 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:21.098308 (+ 384us) server_negotiation.cc:299] Negotiation successful
0620 23:04:21.098668 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":396,"threads_started":1}
I20260620 23:04:21.103883 2996 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:04:21.107580 24104 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ef7aa220c7549de9a3ab312718bdcf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ef7aa220c7549de9a3ab312718bdcf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36173 } }
I20260620 23:04:21.107975 24104 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ef7aa220c7549de9a3ab312718bdcf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:21.108109 24104 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ef7aa220c7549de9a3ab312718bdcf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ef7aa220c7549de9a3ab312718bdcf6, State: Initialized, Role: FOLLOWER
I20260620 23:04:21.108842 24104 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ef7aa220c7549de9a3ab312718bdcf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ef7aa220c7549de9a3ab312718bdcf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36173 } }
I20260620 23:04:21.109126 24104 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ef7aa220c7549de9a3ab312718bdcf6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:04:21.109334 24104 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ef7aa220c7549de9a3ab312718bdcf6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:04:21.109530 24104 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ef7aa220c7549de9a3ab312718bdcf6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:21.112322 24104 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ef7aa220c7549de9a3ab312718bdcf6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ef7aa220c7549de9a3ab312718bdcf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36173 } }
I20260620 23:04:21.112821 24104 leader_election.cc:302] T 00000000000000000000000000000000 P 7ef7aa220c7549de9a3ab312718bdcf6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ef7aa220c7549de9a3ab312718bdcf6; no voters:
I20260620 23:04:21.113504 24104 leader_election.cc:288] T 00000000000000000000000000000000 P 7ef7aa220c7549de9a3ab312718bdcf6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:04:21.113567 24114 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ef7aa220c7549de9a3ab312718bdcf6 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:21.115144 24114 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ef7aa220c7549de9a3ab312718bdcf6 [term 1 LEADER]: Becoming Leader. State: Replica: 7ef7aa220c7549de9a3ab312718bdcf6, State: Running, Role: LEADER
I20260620 23:04:21.115710 24114 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ef7aa220c7549de9a3ab312718bdcf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ef7aa220c7549de9a3ab312718bdcf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36173 } }
I20260620 23:04:21.115998 24104 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ef7aa220c7549de9a3ab312718bdcf6 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:21.118678 24116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ef7aa220c7549de9a3ab312718bdcf6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ef7aa220c7549de9a3ab312718bdcf6. Latest consensus state: current_term: 1 leader_uuid: "7ef7aa220c7549de9a3ab312718bdcf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ef7aa220c7549de9a3ab312718bdcf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36173 } } }
I20260620 23:04:21.119138 24116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ef7aa220c7549de9a3ab312718bdcf6 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:21.119844 24115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ef7aa220c7549de9a3ab312718bdcf6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ef7aa220c7549de9a3ab312718bdcf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ef7aa220c7549de9a3ab312718bdcf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36173 } } }
I20260620 23:04:21.120198 24115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ef7aa220c7549de9a3ab312718bdcf6 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:21.129194 24107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:21.108133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.217:33001 (local address 127.0.0.1:36173)
0620 23:04:21.108315 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:21.108321 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:21.108816 (+ 495us) server_negotiation.cc:407] Connection header received
0620 23:04:21.108927 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:21.108932 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:21.109011 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:21.109118 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:21.110644 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:21.111790 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:21.113115 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:21.114048 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:21.114201 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:21.117832 (+ 3631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:21.117844 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:21.117849 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:21.117906 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:21.122545 (+ 4639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:21.124112 (+ 1567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:21.124125 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:21.124133 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:21.124331 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:21.125303 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:21.125311 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:21.125317 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:21.125938 (+ 621us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:21.128048 (+ 2110us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:21.128311 (+ 263us) server_negotiation.cc:299] Negotiation successful
0620 23:04:21.128556 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260620 23:04:21.133164 24120 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:04:21.139226 24120 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:04:21.148288 24120 catalog_manager.cc:1347] Generated new cluster ID: 82a645d37f264ea6b5eeda7752ba7891
I20260620 23:04:21.148407 24120 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:04:21.165736 24120 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:04:21.167779 24120 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:04:21.177618 24120 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ef7aa220c7549de9a3ab312718bdcf6: Generated new TSK 0
I20260620 23:04:21.178500 24120 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:04:21.191455 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 24037
I20260620 23:04:21.192967 24099 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:04:21.284102 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 24037
W20260620 23:04:21.301728 24022 connection.cc:535] client connection to 127.0.0.1:36173 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36173): Connection reset by peer (error 104)
2026-06-20T23:04:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2057 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:04:21 dist-test-slave-f63b krb5kdc[24136](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:04:21 dist-test-slave-f63b krb5kdc[24136](info): set up 2 sockets
Jun 20 23:04:21 dist-test-slave-f63b krb5kdc[24136](info): commencing operation
krb5kdc: starting...
W20260620 23:04:22.515566 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.002s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 23:04:22 dist-test-slave-f63b krb5kdc[24136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:04:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:04:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:04:22.785708 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45435
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:23.120692 24149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:23.120998 24149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:23.121161 24149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:23.129210 24149 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:23.129328 24149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:23.129417 24149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:23.129519 24149 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:23.129609 24149 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:23.139978 24149 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:23.141434 24149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:23.142822 24149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:23.154776 24157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:23.154778 24154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:23.154817 24155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:23.156159 24149 server_base.cc:1034] running on GCE node
I20260620 23:04:23.157002 24149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:23.158541 24149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:23.159866 24149 hybrid_clock.cc:648] HybridClock initialized: now 1781996663159822 us; error 44 us; skew 500 ppm
I20260620 23:04:23.160240 24149 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:04:23 dist-test-slave-f63b krb5kdc[24136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996663, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:23.163566 24149 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:04:23.165251 24149 webserver.cc:460] Webserver started at http://127.0.0.1:42427/ using document root <none> and password file <none>
I20260620 23:04:23.165998 24149 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:23.166147 24149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:23.166460 24149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:23.169574 24149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "48dc64b79e4a44f0a1dec139865f4505"
format_stamp: "Formatted at 2026-06-20 23:04:23 on dist-test-slave-f63b"
server_key: "b4d64400f9b536233a53786696f9f0b9"
server_key_iv: "a3305197b1a57ecbee917c7702b48096"
server_key_version: "encryptionkey@0"
I20260620 23:04:23.170519 24149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "48dc64b79e4a44f0a1dec139865f4505"
format_stamp: "Formatted at 2026-06-20 23:04:23 on dist-test-slave-f63b"
server_key: "b4d64400f9b536233a53786696f9f0b9"
server_key_iv: "a3305197b1a57ecbee917c7702b48096"
server_key_version: "encryptionkey@0"
I20260620 23:04:23.176121 24149 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260620 23:04:23.179848 24164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:23.181303 24149 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260620 23:04:23.181489 24149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "48dc64b79e4a44f0a1dec139865f4505"
format_stamp: "Formatted at 2026-06-20 23:04:23 on dist-test-slave-f63b"
server_key: "b4d64400f9b536233a53786696f9f0b9"
server_key_iv: "a3305197b1a57ecbee917c7702b48096"
server_key_version: "encryptionkey@0"
I20260620 23:04:23.181699 24149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:23.196688 24149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:23.201045 24149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:23.201346 24149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:23.220872 24149 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45435
I20260620 23:04:23.220927 24215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45435 every 8 connection(s)
I20260620 23:04:23.222388 24149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:04:23.226336 24216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:23.231241 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 24149
I20260620 23:04:23.231438 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:04:23.232002 2996 external_mini_cluster.cc:1428] Setting key 9efc6e2ad39f1c091079524cbcd3da93
I20260620 23:04:23.237936 24216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48dc64b79e4a44f0a1dec139865f4505: Bootstrap starting.
I20260620 23:04:23.241163 24216 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48dc64b79e4a44f0a1dec139865f4505: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:23.242237 24216 log.cc:824] T 00000000000000000000000000000000 P 48dc64b79e4a44f0a1dec139865f4505: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:23.245627 24216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48dc64b79e4a44f0a1dec139865f4505: No bootstrap required, opened a new log
I20260620 23:04:23.245831 24219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:23.233946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.217:42463 (local address 127.0.0.1:45435)
0620 23:04:23.234745 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:23.234789 (+ 44us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:23.234832 (+ 43us) server_negotiation.cc:407] Connection header received
0620 23:04:23.236651 (+ 1819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:23.236676 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:23.237121 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:23.237514 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:23.238077 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.239481 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.240776 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.241586 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.241787 (+ 201us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:23.243158 (+ 1371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:23.243166 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:23.243183 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:23.243841 (+ 658us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:23.243937 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:23.243968 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:23.244126 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:23.244382 (+ 256us) server_negotiation.cc:299] Negotiation successful
0620 23:04:23.244553 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":304,"threads_started":1}
I20260620 23:04:23.249835 2996 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:04:23.254335 24216 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48dc64b79e4a44f0a1dec139865f4505 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48dc64b79e4a44f0a1dec139865f4505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45435 } }
I20260620 23:04:23.254813 24216 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48dc64b79e4a44f0a1dec139865f4505 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:23.254935 24216 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48dc64b79e4a44f0a1dec139865f4505 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48dc64b79e4a44f0a1dec139865f4505, State: Initialized, Role: FOLLOWER
I20260620 23:04:23.255620 24216 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48dc64b79e4a44f0a1dec139865f4505 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48dc64b79e4a44f0a1dec139865f4505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45435 } }
I20260620 23:04:23.255873 24216 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48dc64b79e4a44f0a1dec139865f4505 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:04:23.256042 24216 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48dc64b79e4a44f0a1dec139865f4505 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:04:23.256270 24216 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48dc64b79e4a44f0a1dec139865f4505 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:23.259020 24216 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48dc64b79e4a44f0a1dec139865f4505 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48dc64b79e4a44f0a1dec139865f4505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45435 } }
I20260620 23:04:23.259619 24216 leader_election.cc:302] T 00000000000000000000000000000000 P 48dc64b79e4a44f0a1dec139865f4505 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48dc64b79e4a44f0a1dec139865f4505; no voters:
I20260620 23:04:23.260322 24216 leader_election.cc:288] T 00000000000000000000000000000000 P 48dc64b79e4a44f0a1dec139865f4505 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:04:23.260509 24226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48dc64b79e4a44f0a1dec139865f4505 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:23.261535 24219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:23.254279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.217:44985 (local address 127.0.0.1:45435)
0620 23:04:23.254518 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:23.254528 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:23.255081 (+ 553us) server_negotiation.cc:407] Connection header received
0620 23:04:23.255300 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:23.255304 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:23.255373 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:23.255487 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:23.256076 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.257239 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.258552 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:23.259076 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:23.259219 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:23.260513 (+ 1294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:23.260518 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:23.260523 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:23.260546 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:23.260630 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:23.260656 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:23.260807 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:23.261002 (+ 195us) server_negotiation.cc:299] Negotiation successful
0620 23:04:23.261056 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 23:04:23.263154 24216 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48dc64b79e4a44f0a1dec139865f4505 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:23.263558 24226 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48dc64b79e4a44f0a1dec139865f4505 [term 1 LEADER]: Becoming Leader. State: Replica: 48dc64b79e4a44f0a1dec139865f4505, State: Running, Role: LEADER
I20260620 23:04:23.264161 24226 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48dc64b79e4a44f0a1dec139865f4505 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48dc64b79e4a44f0a1dec139865f4505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45435 } }
I20260620 23:04:23.268603 24227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48dc64b79e4a44f0a1dec139865f4505 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48dc64b79e4a44f0a1dec139865f4505" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48dc64b79e4a44f0a1dec139865f4505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45435 } } }
I20260620 23:04:23.269016 24227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48dc64b79e4a44f0a1dec139865f4505 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:23.269757 24228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48dc64b79e4a44f0a1dec139865f4505 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48dc64b79e4a44f0a1dec139865f4505. Latest consensus state: current_term: 1 leader_uuid: "48dc64b79e4a44f0a1dec139865f4505" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48dc64b79e4a44f0a1dec139865f4505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45435 } } }
I20260620 23:04:23.270036 24228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48dc64b79e4a44f0a1dec139865f4505 [sys.catalog]: This master's current role is: LEADER
W20260620 23:04:23.274070 24240 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 48dc64b79e4a44f0a1dec139865f4505: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:04:23.274183 24240 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:04:23.274319 24241 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:04:23.276538 24241 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:04:23.285475 24241 catalog_manager.cc:1347] Generated new cluster ID: 7e6eb58864b74a9b828c63cb87976b4e
I20260620 23:04:23.285595 24241 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:04:23.313133 24241 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:04:23.315133 24241 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:04:23.323868 24241 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48dc64b79e4a44f0a1dec139865f4505: Generated new TSK 0
I20260620 23:04:23.325023 24241 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:04:23.335704 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 24149
W20260620 23:04:23.337148 24211 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.217:42463
W20260620 23:04:23.338035 2996 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 24149: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260620 23:04:23.338222 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 24149
W20260620 23:04:23.355656 24134 connection.cc:535] client connection to 127.0.0.1:45435 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45435): Connection reset by peer (error 104)
2026-06-20T23:04:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2053 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:04:23 dist-test-slave-f63b krb5kdc[24247](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:04:23 dist-test-slave-f63b krb5kdc[24247](info): set up 2 sockets
Jun 20 23:04:23 dist-test-slave-f63b krb5kdc[24247](info): commencing operation
krb5kdc: starting...
W20260620 23:04:24.455106 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 23:04:24 dist-test-slave-f63b krb5kdc[24247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:04:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:04:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:04:24.726233 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34403
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:25.064473 24260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:25.064785 24260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:25.064968 24260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:25.072515 24260 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:25.072645 24260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:25.072757 24260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:25.072858 24260 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:25.072937 24260 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:25.083376 24260 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:25.084802 24260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:25.086222 24260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:25.098832 24266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:25.098829 24268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:25.099207 24265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:25.099640 24260 server_base.cc:1034] running on GCE node
I20260620 23:04:25.100524 24260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:25.102108 24260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:25.103353 24260 hybrid_clock.cc:648] HybridClock initialized: now 1781996665103304 us; error 47 us; skew 500 ppm
I20260620 23:04:25.103719 24260 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:04:25 dist-test-slave-f63b krb5kdc[24247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996665, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:25.107012 24260 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:04:25.108578 24260 webserver.cc:460] Webserver started at http://127.0.0.1:38645/ using document root <none> and password file <none>
I20260620 23:04:25.109328 24260 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:25.109493 24260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:25.109814 24260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:25.113065 24260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "6494d3275e7e4befb8d6dd2d837bdd5c"
format_stamp: "Formatted at 2026-06-20 23:04:25 on dist-test-slave-f63b"
server_key: "e01972ca19dc2d6953896543eba655c9"
server_key_iv: "5563b78c84cdd314484ddb399f778333"
server_key_version: "encryptionkey@0"
I20260620 23:04:25.113888 24260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "6494d3275e7e4befb8d6dd2d837bdd5c"
format_stamp: "Formatted at 2026-06-20 23:04:25 on dist-test-slave-f63b"
server_key: "e01972ca19dc2d6953896543eba655c9"
server_key_iv: "5563b78c84cdd314484ddb399f778333"
server_key_version: "encryptionkey@0"
I20260620 23:04:25.119382 24260 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260620 23:04:25.123255 24275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:25.124634 24260 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260620 23:04:25.124832 24260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "6494d3275e7e4befb8d6dd2d837bdd5c"
format_stamp: "Formatted at 2026-06-20 23:04:25 on dist-test-slave-f63b"
server_key: "e01972ca19dc2d6953896543eba655c9"
server_key_iv: "5563b78c84cdd314484ddb399f778333"
server_key_version: "encryptionkey@0"
I20260620 23:04:25.125047 24260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:25.131062 24260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:25.135175 24260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:25.135504 24260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:25.154393 24260 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34403
I20260620 23:04:25.154440 24326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34403 every 8 connection(s)
I20260620 23:04:25.155853 24260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:04:25.159641 24327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:25.161901 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 24260
I20260620 23:04:25.162096 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:04:25.162662 2996 external_mini_cluster.cc:1428] Setting key ca3358e033f6074379a34f69c18c7fe3
I20260620 23:04:25.170441 24330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:25.164505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.217:39305 (local address 127.0.0.1:34403)
0620 23:04:25.165269 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:25.165284 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:25.165324 (+ 40us) server_negotiation.cc:407] Connection header received
0620 23:04:25.166980 (+ 1656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:25.167006 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:25.167357 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:25.167760 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:25.168813 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:25.168819 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:25.168832 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:25.168845 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:25.168909 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:25.168937 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:25.169167 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:25.169304 (+ 137us) server_negotiation.cc:299] Negotiation successful
0620 23:04:25.169494 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":321,"threads_started":1}
I20260620 23:04:25.173583 24327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6494d3275e7e4befb8d6dd2d837bdd5c: Bootstrap starting.
I20260620 23:04:25.174494 2996 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:04:25.177397 24327 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6494d3275e7e4befb8d6dd2d837bdd5c: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:25.178611 24327 log.cc:824] T 00000000000000000000000000000000 P 6494d3275e7e4befb8d6dd2d837bdd5c: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:25.181870 24327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6494d3275e7e4befb8d6dd2d837bdd5c: No bootstrap required, opened a new log
I20260620 23:04:25.182139 24330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:25.179435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.217:52667 (local address 127.0.0.1:34403)
0620 23:04:25.179667 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:25.179677 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:25.180177 (+ 500us) server_negotiation.cc:407] Connection header received
0620 23:04:25.180374 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:25.180385 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:25.180439 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:25.180547 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:25.181498 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:25.181503 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:25.181506 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:25.181520 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:25.181559 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:25.181574 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:25.181696 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:25.181791 (+ 95us) server_negotiation.cc:299] Negotiation successful
0620 23:04:25.181854 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260620 23:04:25.190232 24327 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6494d3275e7e4befb8d6dd2d837bdd5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6494d3275e7e4befb8d6dd2d837bdd5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34403 } }
I20260620 23:04:25.190680 24327 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6494d3275e7e4befb8d6dd2d837bdd5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:25.190801 24327 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6494d3275e7e4befb8d6dd2d837bdd5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6494d3275e7e4befb8d6dd2d837bdd5c, State: Initialized, Role: FOLLOWER
I20260620 23:04:25.191534 24327 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6494d3275e7e4befb8d6dd2d837bdd5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6494d3275e7e4befb8d6dd2d837bdd5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34403 } }
I20260620 23:04:25.191846 24327 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6494d3275e7e4befb8d6dd2d837bdd5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:04:25.191992 24327 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6494d3275e7e4befb8d6dd2d837bdd5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:04:25.192148 24327 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6494d3275e7e4befb8d6dd2d837bdd5c [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:25.194317 24327 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6494d3275e7e4befb8d6dd2d837bdd5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6494d3275e7e4befb8d6dd2d837bdd5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34403 } }
I20260620 23:04:25.194867 24327 leader_election.cc:302] T 00000000000000000000000000000000 P 6494d3275e7e4befb8d6dd2d837bdd5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6494d3275e7e4befb8d6dd2d837bdd5c; no voters:
I20260620 23:04:25.195382 24327 leader_election.cc:288] T 00000000000000000000000000000000 P 6494d3275e7e4befb8d6dd2d837bdd5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:04:25.195546 24337 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6494d3275e7e4befb8d6dd2d837bdd5c [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:25.196234 24337 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6494d3275e7e4befb8d6dd2d837bdd5c [term 1 LEADER]: Becoming Leader. State: Replica: 6494d3275e7e4befb8d6dd2d837bdd5c, State: Running, Role: LEADER
I20260620 23:04:25.196753 24337 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6494d3275e7e4befb8d6dd2d837bdd5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6494d3275e7e4befb8d6dd2d837bdd5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34403 } }
I20260620 23:04:25.197479 24327 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6494d3275e7e4befb8d6dd2d837bdd5c [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:25.199734 24339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6494d3275e7e4befb8d6dd2d837bdd5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6494d3275e7e4befb8d6dd2d837bdd5c. Latest consensus state: current_term: 1 leader_uuid: "6494d3275e7e4befb8d6dd2d837bdd5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6494d3275e7e4befb8d6dd2d837bdd5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34403 } } }
I20260620 23:04:25.200232 24339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6494d3275e7e4befb8d6dd2d837bdd5c [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:25.200074 24338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6494d3275e7e4befb8d6dd2d837bdd5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6494d3275e7e4befb8d6dd2d837bdd5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6494d3275e7e4befb8d6dd2d837bdd5c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34403 } } }
I20260620 23:04:25.200351 24338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6494d3275e7e4befb8d6dd2d837bdd5c [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:25.202495 24344 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:04:25.207844 24344 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:04:25.216907 24344 catalog_manager.cc:1347] Generated new cluster ID: dc2272d704484678884bf6e8295e4409
I20260620 23:04:25.217092 24344 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:04:25.252511 24344 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:04:25.252954 24344 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:04:25.261523 24344 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6494d3275e7e4befb8d6dd2d837bdd5c: Generated new TSK 0
I20260620 23:04:25.262632 24344 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:04:25.276013 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 24260
W20260620 23:04:25.277235 24322 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.217:39305
W20260620 23:04:25.278177 2996 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 24260: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260620 23:04:25.278333 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 24260
2026-06-20T23:04:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1940 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:04:25 dist-test-slave-f63b krb5kdc[24358](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:04:25 dist-test-slave-f63b krb5kdc[24358](info): set up 2 sockets
Jun 20 23:04:25 dist-test-slave-f63b krb5kdc[24358](info): commencing operation
krb5kdc: starting...
W20260620 23:04:26.391067 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 23:04:26 dist-test-slave-f63b krb5kdc[24358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:04:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:04:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:04:26.651485 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39623
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39623
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:26.987847 24371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:26.988229 24371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:26.988410 24371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:26.995810 24371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:26.995988 24371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:26.996106 24371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:26.996227 24371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:26.996325 24371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:27.006558 24371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39623
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:27.008042 24371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:27.009398 24371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:27.021531 24376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:27.021615 24379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:27.021531 24377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:27.022861 24371 server_base.cc:1034] running on GCE node
I20260620 23:04:27.023761 24371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:27.025362 24371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:27.026556 24371 hybrid_clock.cc:648] HybridClock initialized: now 1781996667026496 us; error 53 us; skew 500 ppm
I20260620 23:04:27.026958 24371 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:04:27 dist-test-slave-f63b krb5kdc[24358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996667, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:27.034242 24371 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:04:27.035995 24371 webserver.cc:460] Webserver started at http://127.0.0.1:39003/ using document root <none> and password file <none>
I20260620 23:04:27.036747 24371 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:27.036881 24371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:27.037206 24371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:27.040387 24371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "767820ad195e427385feaef8cb67b2c2"
format_stamp: "Formatted at 2026-06-20 23:04:27 on dist-test-slave-f63b"
server_key: "32a744b162fa7b646ad1c2f6425736b8"
server_key_iv: "324bc8580d679cb84d8f2a6c7d0934a9"
server_key_version: "encryptionkey@0"
I20260620 23:04:27.041167 24371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "767820ad195e427385feaef8cb67b2c2"
format_stamp: "Formatted at 2026-06-20 23:04:27 on dist-test-slave-f63b"
server_key: "32a744b162fa7b646ad1c2f6425736b8"
server_key_iv: "324bc8580d679cb84d8f2a6c7d0934a9"
server_key_version: "encryptionkey@0"
I20260620 23:04:27.046628 24371 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260620 23:04:27.050096 24386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:27.051483 24371 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 23:04:27.051679 24371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "767820ad195e427385feaef8cb67b2c2"
format_stamp: "Formatted at 2026-06-20 23:04:27 on dist-test-slave-f63b"
server_key: "32a744b162fa7b646ad1c2f6425736b8"
server_key_iv: "324bc8580d679cb84d8f2a6c7d0934a9"
server_key_version: "encryptionkey@0"
I20260620 23:04:27.051847 24371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:27.076368 24371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:27.080608 24371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:27.080902 24371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:27.100847 24371 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39623
I20260620 23:04:27.100903 24437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39623 every 8 connection(s)
I20260620 23:04:27.102358 24371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:04:27.106165 24438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:27.106724 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 24371
I20260620 23:04:27.106912 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:04:27.107487 2996 external_mini_cluster.cc:1428] Setting key 188d6e9b48d0514e40fbe8dc687d1c92
I20260620 23:04:27.120491 24438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 767820ad195e427385feaef8cb67b2c2: Bootstrap starting.
Jun 20 23:04:27 dist-test-slave-f63b krb5kdc[24358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260620 23:04:27.124212 24438 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 767820ad195e427385feaef8cb67b2c2: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:27.125589 24438 log.cc:824] T 00000000000000000000000000000000 P 767820ad195e427385feaef8cb67b2c2: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:27.128562 24438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 767820ad195e427385feaef8cb67b2c2: No bootstrap required, opened a new log
I20260620 23:04:27.131500 24441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:27.109348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.217:37935 (local address 127.0.0.1:39623)
0620 23:04:27.110150 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:27.110161 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:27.110205 (+ 44us) server_negotiation.cc:407] Connection header received
0620 23:04:27.112467 (+ 2262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:27.112489 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:27.112900 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:27.113228 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:27.114954 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:27.116277 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:27.118134 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:27.118868 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:27.118999 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:27.123804 (+ 4805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:27.123811 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:27.123827 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:27.123864 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:27.127168 (+ 3304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:27.128031 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:27.128037 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:27.128043 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:27.128200 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:27.128681 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:27.128686 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:27.128689 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:27.129201 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:27.129540 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:27.130075 (+ 535us) server_negotiation.cc:299] Negotiation successful
0620 23:04:27.130404 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":367,"threads_started":1}
I20260620 23:04:27.135403 2996 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:04:27.136873 24438 raft_consensus.cc:348] T 00000000000000000000000000000000 P 767820ad195e427385feaef8cb67b2c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "767820ad195e427385feaef8cb67b2c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39623 } }
I20260620 23:04:27.137245 24438 raft_consensus.cc:374] T 00000000000000000000000000000000 P 767820ad195e427385feaef8cb67b2c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:27.137362 24438 raft_consensus.cc:729] T 00000000000000000000000000000000 P 767820ad195e427385feaef8cb67b2c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 767820ad195e427385feaef8cb67b2c2, State: Initialized, Role: FOLLOWER
I20260620 23:04:27.137992 24438 consensus_queue.cc:260] T 00000000000000000000000000000000 P 767820ad195e427385feaef8cb67b2c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "767820ad195e427385feaef8cb67b2c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39623 } }
I20260620 23:04:27.138267 24438 raft_consensus.cc:388] T 00000000000000000000000000000000 P 767820ad195e427385feaef8cb67b2c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:04:27.138441 24438 raft_consensus.cc:482] T 00000000000000000000000000000000 P 767820ad195e427385feaef8cb67b2c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:04:27.138690 24438 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 767820ad195e427385feaef8cb67b2c2 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:27.141448 24438 raft_consensus.cc:504] T 00000000000000000000000000000000 P 767820ad195e427385feaef8cb67b2c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "767820ad195e427385feaef8cb67b2c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39623 } }
I20260620 23:04:27.141928 24438 leader_election.cc:302] T 00000000000000000000000000000000 P 767820ad195e427385feaef8cb67b2c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 767820ad195e427385feaef8cb67b2c2; no voters:
I20260620 23:04:27.142573 24438 leader_election.cc:288] T 00000000000000000000000000000000 P 767820ad195e427385feaef8cb67b2c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:04:27.142730 24448 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 767820ad195e427385feaef8cb67b2c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:27.143978 24448 raft_consensus.cc:686] T 00000000000000000000000000000000 P 767820ad195e427385feaef8cb67b2c2 [term 1 LEADER]: Becoming Leader. State: Replica: 767820ad195e427385feaef8cb67b2c2, State: Running, Role: LEADER
I20260620 23:04:27.144562 24448 consensus_queue.cc:237] T 00000000000000000000000000000000 P 767820ad195e427385feaef8cb67b2c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "767820ad195e427385feaef8cb67b2c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39623 } }
I20260620 23:04:27.144850 24438 sys_catalog.cc:564] T 00000000000000000000000000000000 P 767820ad195e427385feaef8cb67b2c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:27.147296 24449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 767820ad195e427385feaef8cb67b2c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "767820ad195e427385feaef8cb67b2c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "767820ad195e427385feaef8cb67b2c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39623 } } }
I20260620 23:04:27.147703 24449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 767820ad195e427385feaef8cb67b2c2 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:27.150128 24450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 767820ad195e427385feaef8cb67b2c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 767820ad195e427385feaef8cb67b2c2. Latest consensus state: current_term: 1 leader_uuid: "767820ad195e427385feaef8cb67b2c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "767820ad195e427385feaef8cb67b2c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39623 } } }
I20260620 23:04:27.150497 24450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 767820ad195e427385feaef8cb67b2c2 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:27.152637 24453 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:04:27.157460 24441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:27.139870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.217:44543 (local address 127.0.0.1:39623)
0620 23:04:27.140061 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:27.140067 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:27.140561 (+ 494us) server_negotiation.cc:407] Connection header received
0620 23:04:27.140682 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:27.140687 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:27.140760 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:27.140896 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:27.142399 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:27.143710 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:27.145526 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:27.146328 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:27.147218 (+ 890us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:27.150220 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:27.150226 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:27.150230 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:27.150278 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:27.153651 (+ 3373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:27.154400 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:27.154406 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:27.154413 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:27.154535 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:27.155261 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:27.155268 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:27.155277 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:27.155756 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:27.155985 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:27.156863 (+ 878us) server_negotiation.cc:299] Negotiation successful
0620 23:04:27.157061 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260620 23:04:27.159407 24453 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:04:27.170077 24453 catalog_manager.cc:1347] Generated new cluster ID: 6edbb34e6e3b467ca81a46d3b1fc427f
I20260620 23:04:27.170336 24453 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:04:27.192209 24453 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:04:27.194108 24453 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:04:27.203802 24453 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 767820ad195e427385feaef8cb67b2c2: Generated new TSK 0
I20260620 23:04:27.204882 24453 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:04:27.212222 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 24371
I20260620 23:04:27.213541 24433 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:04:27.307051 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 24371
W20260620 23:04:27.324589 24356 connection.cc:535] client connection to 127.0.0.1:39623 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39623): Connection reset by peer (error 104)
2026-06-20T23:04:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2028 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:04:27 dist-test-slave-f63b krb5kdc[24470](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:04:27 dist-test-slave-f63b krb5kdc[24470](info): set up 2 sockets
Jun 20 23:04:27 dist-test-slave-f63b krb5kdc[24470](info): commencing operation
krb5kdc: starting...
W20260620 23:04:28.423076 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.002s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 23:04:28 dist-test-slave-f63b krb5kdc[24470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:04:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:04:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:04:28.689855 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39047
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39047
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:29.027006 24483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:29.027318 24483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:29.027413 24483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:29.034855 24483 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:29.034982 24483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:29.035028 24483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:29.035089 24483 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:29.035133 24483 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:29.045166 24483 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39047
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:29.046492 24483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:29.047928 24483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:29.060062 24488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:29.060140 24489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:29.060091 24491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:29.061318 24483 server_base.cc:1034] running on GCE node
I20260620 23:04:29.062191 24483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:29.063880 24483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:29.065184 24483 hybrid_clock.cc:648] HybridClock initialized: now 1781996669065139 us; error 29 us; skew 500 ppm
I20260620 23:04:29.065507 24483 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:04:29 dist-test-slave-f63b krb5kdc[24470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996669, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:29.068758 24483 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:04:29.070245 24483 webserver.cc:460] Webserver started at http://127.0.0.1:33933/ using document root <none> and password file <none>
I20260620 23:04:29.070983 24483 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:29.071087 24483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:29.071336 24483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:29.074383 24483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "f3497bff70b445da9e34e9545c80629f"
format_stamp: "Formatted at 2026-06-20 23:04:29 on dist-test-slave-f63b"
server_key: "de9a7fe5517d0f8b470ac6c2e593a50b"
server_key_iv: "c5d03a9c6176c903e5b436d5ac68d25f"
server_key_version: "encryptionkey@0"
I20260620 23:04:29.075259 24483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "f3497bff70b445da9e34e9545c80629f"
format_stamp: "Formatted at 2026-06-20 23:04:29 on dist-test-slave-f63b"
server_key: "de9a7fe5517d0f8b470ac6c2e593a50b"
server_key_iv: "c5d03a9c6176c903e5b436d5ac68d25f"
server_key_version: "encryptionkey@0"
I20260620 23:04:29.080878 24483 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260620 23:04:29.084674 24498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:29.086158 24483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 23:04:29.086304 24483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "f3497bff70b445da9e34e9545c80629f"
format_stamp: "Formatted at 2026-06-20 23:04:29 on dist-test-slave-f63b"
server_key: "de9a7fe5517d0f8b470ac6c2e593a50b"
server_key_iv: "c5d03a9c6176c903e5b436d5ac68d25f"
server_key_version: "encryptionkey@0"
I20260620 23:04:29.086481 24483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:29.106352 24483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:29.110811 24483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:29.111066 24483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:29.130412 24483 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39047
I20260620 23:04:29.130435 24549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39047 every 8 connection(s)
I20260620 23:04:29.131884 24483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:04:29.135946 24550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:29.139233 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 24483
I20260620 23:04:29.139405 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:04:29.139861 2996 external_mini_cluster.cc:1428] Setting key f4b055cf7b5725a16d20ece8cfb98f21
I20260620 23:04:29.149787 24550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3497bff70b445da9e34e9545c80629f: Bootstrap starting.
I20260620 23:04:29.153887 24550 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3497bff70b445da9e34e9545c80629f: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:29.155318 24550 log.cc:824] T 00000000000000000000000000000000 P f3497bff70b445da9e34e9545c80629f: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:29.155705 24553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:29.141855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.217:35701 (local address 127.0.0.1:39047)
0620 23:04:29.142711 (+ 856us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:29.142750 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:29.142804 (+ 54us) server_negotiation.cc:407] Connection header received
0620 23:04:29.145176 (+ 2372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:29.145211 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:29.145812 (+ 601us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:29.146265 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:29.146861 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:29.148881 (+ 2020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:29.150305 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:29.151356 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:29.151563 (+ 207us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:29.152749 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:29.152759 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:29.152777 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:29.153503 (+ 726us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:29.153598 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:29.153634 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:29.153805 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:29.154095 (+ 290us) server_negotiation.cc:299] Negotiation successful
0620 23:04:29.154295 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":321,"threads_started":1}
I20260620 23:04:29.159219 24550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3497bff70b445da9e34e9545c80629f: No bootstrap required, opened a new log
I20260620 23:04:29.160569 2996 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:04:29.170007 24550 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3497bff70b445da9e34e9545c80629f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3497bff70b445da9e34e9545c80629f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39047 } }
I20260620 23:04:29.170531 24550 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3497bff70b445da9e34e9545c80629f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:29.170679 24550 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3497bff70b445da9e34e9545c80629f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3497bff70b445da9e34e9545c80629f, State: Initialized, Role: FOLLOWER
I20260620 23:04:29.171394 24550 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3497bff70b445da9e34e9545c80629f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3497bff70b445da9e34e9545c80629f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39047 } }
I20260620 23:04:29.171679 24550 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3497bff70b445da9e34e9545c80629f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:04:29.171849 24550 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3497bff70b445da9e34e9545c80629f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:04:29.172053 24550 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3497bff70b445da9e34e9545c80629f [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:29.173105 24553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:29.166063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.217:40057 (local address 127.0.0.1:39047)
0620 23:04:29.166252 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:29.166259 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:29.166755 (+ 496us) server_negotiation.cc:407] Connection header received
0620 23:04:29.166906 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:29.166913 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:29.167010 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:29.167128 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:29.167651 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:29.168882 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:29.170200 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:29.170903 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:29.171035 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:29.172246 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:29.172252 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:29.172257 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:29.172280 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:29.172362 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:29.172390 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:29.172515 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:29.172690 (+ 175us) server_negotiation.cc:299] Negotiation successful
0620 23:04:29.172756 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 23:04:29.175225 24550 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3497bff70b445da9e34e9545c80629f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3497bff70b445da9e34e9545c80629f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39047 } }
I20260620 23:04:29.175715 24550 leader_election.cc:302] T 00000000000000000000000000000000 P f3497bff70b445da9e34e9545c80629f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3497bff70b445da9e34e9545c80629f; no voters:
I20260620 23:04:29.176237 24550 leader_election.cc:288] T 00000000000000000000000000000000 P f3497bff70b445da9e34e9545c80629f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:04:29.176378 24560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3497bff70b445da9e34e9545c80629f [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:29.177495 24560 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3497bff70b445da9e34e9545c80629f [term 1 LEADER]: Becoming Leader. State: Replica: f3497bff70b445da9e34e9545c80629f, State: Running, Role: LEADER
I20260620 23:04:29.178143 24560 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3497bff70b445da9e34e9545c80629f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3497bff70b445da9e34e9545c80629f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39047 } }
I20260620 23:04:29.178699 24550 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3497bff70b445da9e34e9545c80629f [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:29.181736 24562 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3497bff70b445da9e34e9545c80629f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3497bff70b445da9e34e9545c80629f. Latest consensus state: current_term: 1 leader_uuid: "f3497bff70b445da9e34e9545c80629f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3497bff70b445da9e34e9545c80629f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39047 } } }
I20260620 23:04:29.182201 24562 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3497bff70b445da9e34e9545c80629f [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:29.181711 24561 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3497bff70b445da9e34e9545c80629f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3497bff70b445da9e34e9545c80629f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3497bff70b445da9e34e9545c80629f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39047 } } }
I20260620 23:04:29.182842 24561 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3497bff70b445da9e34e9545c80629f [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:29.184228 24567 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:04:29.189997 24567 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:04:29.199728 24567 catalog_manager.cc:1347] Generated new cluster ID: 04146d389f87418eb92d499a6251787f
I20260620 23:04:29.199865 24567 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:04:29.215143 24567 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:04:29.217077 24567 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:04:29.225996 24567 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3497bff70b445da9e34e9545c80629f: Generated new TSK 0
I20260620 23:04:29.227135 24567 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:04:29.241698 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 24483
W20260620 23:04:29.243013 24545 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.217:35701
W20260620 23:04:29.243767 2996 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 24483: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260620 23:04:29.243922 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 24483
W20260620 23:04:29.265797 24468 connection.cc:535] client connection to 127.0.0.1:39047 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39047): Connection reset by peer (error 104)
2026-06-20T23:04:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1940 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:04:29 dist-test-slave-f63b krb5kdc[24581](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:04:29 dist-test-slave-f63b krb5kdc[24581](info): set up 2 sockets
Jun 20 23:04:29 dist-test-slave-f63b krb5kdc[24581](info): commencing operation
krb5kdc: starting...
W20260620 23:04:30.351044 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.006s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 23:04:30 dist-test-slave-f63b krb5kdc[24581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:04:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:04:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260620 23:04:30.606182 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36005
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:30.940387 24594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:30.940724 24594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:30.940907 24594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:30.948167 24594 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:30.948351 24594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:30.948467 24594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:30.948577 24594 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:30.948666 24594 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:30.958742 24594 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36005
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996318807807-2996-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:30.960172 24594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:30.961529 24594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:30.973541 24600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:30.973613 24599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:30.973611 24602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:30.974352 24594 server_base.cc:1034] running on GCE node
I20260620 23:04:30.975248 24594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:30.976713 24594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:30.977916 24594 hybrid_clock.cc:648] HybridClock initialized: now 1781996670977853 us; error 57 us; skew 500 ppm
I20260620 23:04:30.978272 24594 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:04:30 dist-test-slave-f63b krb5kdc[24581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996670, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:30.981630 24594 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260620 23:04:30.983197 24594 webserver.cc:460] Webserver started at http://127.0.0.1:40155/ using document root <none> and password file <none>
I20260620 23:04:30.983884 24594 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:30.984026 24594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:30.984351 24594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:30.987433 24594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "2843e71e9e4e4ec19fae6c24e404cfba"
format_stamp: "Formatted at 2026-06-20 23:04:30 on dist-test-slave-f63b"
server_key: "6ee4cf6c27220669239b210da689475f"
server_key_iv: "f5678465b530f1988ed397b2594ed91e"
server_key_version: "encryptionkey@0"
I20260620 23:04:30.988256 24594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "2843e71e9e4e4ec19fae6c24e404cfba"
format_stamp: "Formatted at 2026-06-20 23:04:30 on dist-test-slave-f63b"
server_key: "6ee4cf6c27220669239b210da689475f"
server_key_iv: "f5678465b530f1988ed397b2594ed91e"
server_key_version: "encryptionkey@0"
I20260620 23:04:30.993592 24594 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260620 23:04:30.997068 24609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:30.998382 24594 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 23:04:30.998610 24594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "2843e71e9e4e4ec19fae6c24e404cfba"
format_stamp: "Formatted at 2026-06-20 23:04:30 on dist-test-slave-f63b"
server_key: "6ee4cf6c27220669239b210da689475f"
server_key_iv: "f5678465b530f1988ed397b2594ed91e"
server_key_version: "encryptionkey@0"
I20260620 23:04:30.998811 24594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:31.004623 24594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:31.008960 24594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:31.009274 24594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:31.028254 24594 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36005
I20260620 23:04:31.028302 24660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36005 every 8 connection(s)
I20260620 23:04:31.029708 24594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:04:31.031384 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 24594
I20260620 23:04:31.031567 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:04:31.032132 2996 external_mini_cluster.cc:1428] Setting key 44cee5460d082c4309b10b278ca36d75
I20260620 23:04:31.034344 24661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:31.039897 24664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:31.033981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.217:48393 (local address 127.0.0.1:36005)
0620 23:04:31.034843 (+ 862us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:31.034855 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:31.034897 (+ 42us) server_negotiation.cc:407] Connection header received
0620 23:04:31.036464 (+ 1567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:31.036493 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:31.036831 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:31.037244 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:31.038251 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:31.038260 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:31.038272 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:31.038285 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:31.038356 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:31.038378 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:31.038659 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:31.038803 (+ 144us) server_negotiation.cc:299] Negotiation successful
0620 23:04:31.038991 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":312,"threads_started":1}
I20260620 23:04:31.043427 2996 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:04:31.045560 24661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2843e71e9e4e4ec19fae6c24e404cfba: Bootstrap starting.
I20260620 23:04:31.049254 24661 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2843e71e9e4e4ec19fae6c24e404cfba: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:31.050223 24664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:31.047818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.217:36883 (local address 127.0.0.1:36005)
0620 23:04:31.048018 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:31.048025 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:31.048342 (+ 317us) server_negotiation.cc:407] Connection header received
0620 23:04:31.048514 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:31.048518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:31.048561 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:31.048639 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:31.049556 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:31.049559 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:31.049563 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:31.049576 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:31.049620 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:31.049635 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:31.049729 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:31.049876 (+ 147us) server_negotiation.cc:299] Negotiation successful
0620 23:04:31.049922 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260620 23:04:31.050630 24661 log.cc:824] T 00000000000000000000000000000000 P 2843e71e9e4e4ec19fae6c24e404cfba: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:31.053834 24661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2843e71e9e4e4ec19fae6c24e404cfba: No bootstrap required, opened a new log
I20260620 23:04:31.061355 24661 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2843e71e9e4e4ec19fae6c24e404cfba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2843e71e9e4e4ec19fae6c24e404cfba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36005 } }
I20260620 23:04:31.061731 24661 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2843e71e9e4e4ec19fae6c24e404cfba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:31.061849 24661 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2843e71e9e4e4ec19fae6c24e404cfba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2843e71e9e4e4ec19fae6c24e404cfba, State: Initialized, Role: FOLLOWER
I20260620 23:04:31.062382 24661 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2843e71e9e4e4ec19fae6c24e404cfba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2843e71e9e4e4ec19fae6c24e404cfba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36005 } }
I20260620 23:04:31.062639 24661 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2843e71e9e4e4ec19fae6c24e404cfba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:04:31.062788 24661 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2843e71e9e4e4ec19fae6c24e404cfba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:04:31.062966 24661 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2843e71e9e4e4ec19fae6c24e404cfba [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:31.065460 24661 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2843e71e9e4e4ec19fae6c24e404cfba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2843e71e9e4e4ec19fae6c24e404cfba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36005 } }
I20260620 23:04:31.066004 24661 leader_election.cc:302] T 00000000000000000000000000000000 P 2843e71e9e4e4ec19fae6c24e404cfba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2843e71e9e4e4ec19fae6c24e404cfba; no voters:
I20260620 23:04:31.066663 24661 leader_election.cc:288] T 00000000000000000000000000000000 P 2843e71e9e4e4ec19fae6c24e404cfba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:04:31.066790 24671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2843e71e9e4e4ec19fae6c24e404cfba [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:31.067515 24671 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2843e71e9e4e4ec19fae6c24e404cfba [term 1 LEADER]: Becoming Leader. State: Replica: 2843e71e9e4e4ec19fae6c24e404cfba, State: Running, Role: LEADER
I20260620 23:04:31.068073 24671 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2843e71e9e4e4ec19fae6c24e404cfba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2843e71e9e4e4ec19fae6c24e404cfba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36005 } }
I20260620 23:04:31.068734 24661 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2843e71e9e4e4ec19fae6c24e404cfba [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:31.071035 24672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2843e71e9e4e4ec19fae6c24e404cfba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2843e71e9e4e4ec19fae6c24e404cfba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2843e71e9e4e4ec19fae6c24e404cfba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36005 } } }
I20260620 23:04:31.071452 24672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2843e71e9e4e4ec19fae6c24e404cfba [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:31.074909 24673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2843e71e9e4e4ec19fae6c24e404cfba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2843e71e9e4e4ec19fae6c24e404cfba. Latest consensus state: current_term: 1 leader_uuid: "2843e71e9e4e4ec19fae6c24e404cfba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2843e71e9e4e4ec19fae6c24e404cfba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36005 } } }
I20260620 23:04:31.075160 24673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2843e71e9e4e4ec19fae6c24e404cfba [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:31.076140 24680 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:04:31.080392 24680 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:04:31.088994 24680 catalog_manager.cc:1347] Generated new cluster ID: 42449dde97b4454fa231ba9804311a0a
I20260620 23:04:31.089111 24680 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:04:31.111807 24680 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:04:31.112262 24680 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:04:31.122445 24680 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2843e71e9e4e4ec19fae6c24e404cfba: Generated new TSK 0
I20260620 23:04:31.123317 24680 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:04:31.129604 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 24594
W20260620 23:04:31.130861 24656 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.217:48393
W20260620 23:04:31.131614 2996 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 24594: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260620 23:04:31.131762 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 24594
2026-06-20T23:04:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1883 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:04:31 dist-test-slave-f63b krb5kdc[24692](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:04:31 dist-test-slave-f63b krb5kdc[24692](info): set up 2 sockets
Jun 20 23:04:31 dist-test-slave-f63b krb5kdc[24692](info): commencing operation
krb5kdc: starting...
W20260620 23:04:32.251030 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.002s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 23:04:32 dist-test-slave-f63b krb5kdc[24692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:04:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:04:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
I20260620 23:04:32.510317 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:45443
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:37805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:45443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:32.848910 24705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:32.849299 24705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:32.849488 24705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:32.856941 24705 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:32.857126 24705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:32.857215 24705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:32.857311 24705 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:32.857384 24705 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:32.868620 24705 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:37805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:45443
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:45443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:32.870131 24705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:32.871531 24705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:32.883984 24710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:32.883919 24713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:32.883945 24711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:32.885344 24705 server_base.cc:1034] running on GCE node
I20260620 23:04:32.886190 24705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:32.887806 24705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:32.889065 24705 hybrid_clock.cc:648] HybridClock initialized: now 1781996672889004 us; error 54 us; skew 500 ppm
I20260620 23:04:32.889433 24705 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:04:32 dist-test-slave-f63b krb5kdc[24692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996672, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:32.897094 24705 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 23:04:32.898802 24705 webserver.cc:460] Webserver started at http://127.2.237.62:39701/ using document root <none> and password file <none>
I20260620 23:04:32.899534 24705 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:32.899681 24705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:32.899992 24705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:32.903110 24705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "d921bcd9dbc54de8a8d2971436538361"
format_stamp: "Formatted at 2026-06-20 23:04:32 on dist-test-slave-f63b"
server_key: "d88f58893f0bc91541d52e9dd5e02417"
server_key_iv: "81b05fd60f0edc78914be7105e30140f"
server_key_version: "encryptionkey@0"
I20260620 23:04:32.903875 24705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "d921bcd9dbc54de8a8d2971436538361"
format_stamp: "Formatted at 2026-06-20 23:04:32 on dist-test-slave-f63b"
server_key: "d88f58893f0bc91541d52e9dd5e02417"
server_key_iv: "81b05fd60f0edc78914be7105e30140f"
server_key_version: "encryptionkey@0"
I20260620 23:04:32.909206 24705 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260620 23:04:32.912667 24720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:32.914019 24705 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 23:04:32.914261 24705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "d921bcd9dbc54de8a8d2971436538361"
format_stamp: "Formatted at 2026-06-20 23:04:32 on dist-test-slave-f63b"
server_key: "d88f58893f0bc91541d52e9dd5e02417"
server_key_iv: "81b05fd60f0edc78914be7105e30140f"
server_key_version: "encryptionkey@0"
I20260620 23:04:32.914461 24705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:32.949591 24705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:32.954245 24705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:32.954559 24705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:32.974014 24705 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:45443
I20260620 23:04:32.974027 24771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:45443 every 8 connection(s)
I20260620 23:04:32.975517 24705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:04:32.976774 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 24705
I20260620 23:04:32.977008 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:04:32.977461 2996 external_mini_cluster.cc:1428] Setting key f2a572a31521e33f6bff04b7ffca0e3d
I20260620 23:04:32.980101 24772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 20 23:04:32 dist-test-slave-f63b krb5kdc[24692](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:04:32.993110 24772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d921bcd9dbc54de8a8d2971436538361: Bootstrap starting.
I20260620 23:04:32.997601 24772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d921bcd9dbc54de8a8d2971436538361: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:32.998941 24772 log.cc:824] T 00000000000000000000000000000000 P d921bcd9dbc54de8a8d2971436538361: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:33.002382 24772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d921bcd9dbc54de8a8d2971436538361: No bootstrap required, opened a new log
I20260620 23:04:33.002661 24775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:32.979350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46259 (local address 127.2.237.62:45443)
0620 23:04:32.980167 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:32.980183 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:32.980234 (+ 51us) server_negotiation.cc:407] Connection header received
0620 23:04:32.982540 (+ 2306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:32.982595 (+ 55us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:32.983136 (+ 541us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:32.983588 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:32.984898 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:32.987017 (+ 2119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:32.988311 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:32.989330 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:32.989525 (+ 195us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:32.993669 (+ 4144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:32.993681 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:32.993706 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:32.993757 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:32.997846 (+ 4089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:32.998750 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:32.998761 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:32.998769 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:32.999008 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:32.999606 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:32.999612 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:32.999614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:33.000182 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:33.000526 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:33.000954 (+ 428us) server_negotiation.cc:299] Negotiation successful
0620 23:04:33.001288 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":336,"threads_started":1}
I20260620 23:04:33.006558 2996 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:04:33.010819 24772 raft_consensus.cc:348] T 00000000000000000000000000000000 P d921bcd9dbc54de8a8d2971436538361 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d921bcd9dbc54de8a8d2971436538361" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 45443 } }
I20260620 23:04:33.011396 24772 raft_consensus.cc:374] T 00000000000000000000000000000000 P d921bcd9dbc54de8a8d2971436538361 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:33.011504 24772 raft_consensus.cc:729] T 00000000000000000000000000000000 P d921bcd9dbc54de8a8d2971436538361 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d921bcd9dbc54de8a8d2971436538361, State: Initialized, Role: FOLLOWER
I20260620 23:04:33.012255 24772 consensus_queue.cc:260] T 00000000000000000000000000000000 P d921bcd9dbc54de8a8d2971436538361 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d921bcd9dbc54de8a8d2971436538361" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 45443 } }
I20260620 23:04:33.012506 24772 raft_consensus.cc:388] T 00000000000000000000000000000000 P d921bcd9dbc54de8a8d2971436538361 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:04:33.012687 24772 raft_consensus.cc:482] T 00000000000000000000000000000000 P d921bcd9dbc54de8a8d2971436538361 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:04:33.012868 24772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d921bcd9dbc54de8a8d2971436538361 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:33.015753 24772 raft_consensus.cc:504] T 00000000000000000000000000000000 P d921bcd9dbc54de8a8d2971436538361 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d921bcd9dbc54de8a8d2971436538361" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 45443 } }
I20260620 23:04:33.016286 24772 leader_election.cc:302] T 00000000000000000000000000000000 P d921bcd9dbc54de8a8d2971436538361 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d921bcd9dbc54de8a8d2971436538361; no voters:
I20260620 23:04:33.016994 24772 leader_election.cc:288] T 00000000000000000000000000000000 P d921bcd9dbc54de8a8d2971436538361 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:04:33.017143 24782 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d921bcd9dbc54de8a8d2971436538361 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:33.017971 24782 raft_consensus.cc:686] T 00000000000000000000000000000000 P d921bcd9dbc54de8a8d2971436538361 [term 1 LEADER]: Becoming Leader. State: Replica: d921bcd9dbc54de8a8d2971436538361, State: Running, Role: LEADER
I20260620 23:04:33.018716 24782 consensus_queue.cc:237] T 00000000000000000000000000000000 P d921bcd9dbc54de8a8d2971436538361 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d921bcd9dbc54de8a8d2971436538361" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 45443 } }
I20260620 23:04:33.020766 24782 sys_catalog.cc:455] T 00000000000000000000000000000000 P d921bcd9dbc54de8a8d2971436538361 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d921bcd9dbc54de8a8d2971436538361. Latest consensus state: current_term: 1 leader_uuid: "d921bcd9dbc54de8a8d2971436538361" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d921bcd9dbc54de8a8d2971436538361" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 45443 } } }
I20260620 23:04:33.022379 24782 sys_catalog.cc:458] T 00000000000000000000000000000000 P d921bcd9dbc54de8a8d2971436538361 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:33.021996 24784 sys_catalog.cc:455] T 00000000000000000000000000000000 P d921bcd9dbc54de8a8d2971436538361 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d921bcd9dbc54de8a8d2971436538361" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d921bcd9dbc54de8a8d2971436538361" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 45443 } } }
I20260620 23:04:33.022804 24784 sys_catalog.cc:458] T 00000000000000000000000000000000 P d921bcd9dbc54de8a8d2971436538361 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:33.025974 24772 sys_catalog.cc:564] T 00000000000000000000000000000000 P d921bcd9dbc54de8a8d2971436538361 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:33.026289 24786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:04:33.029311 24775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:33.010990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57881 (local address 127.2.237.62:45443)
0620 23:04:33.011159 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:33.011163 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:33.011778 (+ 615us) server_negotiation.cc:407] Connection header received
0620 23:04:33.011964 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:33.011969 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:33.012046 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:33.012162 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:33.013818 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:33.015035 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:33.016396 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:33.017175 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:33.017330 (+ 155us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:33.021018 (+ 3688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:33.021025 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:33.021029 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:33.021076 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:33.025292 (+ 4216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:33.026274 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:33.026280 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:33.026283 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:33.026497 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:33.027087 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:33.027094 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:33.027101 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:33.027559 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:33.027967 (+ 408us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:33.028541 (+ 574us) server_negotiation.cc:299] Negotiation successful
0620 23:04:33.028781 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260620 23:04:33.035161 24786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:04:33.045322 24786 catalog_manager.cc:1347] Generated new cluster ID: 11fa2c79b1c048ac8e4f5ece242a7ba4
I20260620 23:04:33.045461 24786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:04:33.073472 24786 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:04:33.074934 24786 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:04:33.083662 24786 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d921bcd9dbc54de8a8d2971436538361: Generated new TSK 0
I20260620 23:04:33.084806 24786 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:04:33.097358 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 24705
I20260620 23:04:33.098706 24767 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:04:33.192293 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 24705
W20260620 23:04:33.209630 24690 connection.cc:535] client connection to 127.2.237.62:45443 recv error: Network error: failed to read from TLS socket (remote: 127.2.237.62:45443): Connection reset by peer (error 104)
2026-06-20T23:04:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2060 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:04:33 dist-test-slave-f63b krb5kdc[24804](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:04:33 dist-test-slave-f63b krb5kdc[24804](info): set up 2 sockets
Jun 20 23:04:33 dist-test-slave-f63b krb5kdc[24804](info): commencing operation
krb5kdc: starting...
W20260620 23:04:34.303112 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.002s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 23:04:34 dist-test-slave-f63b krb5kdc[24804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:04:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:04:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
I20260620 23:04:34.564045 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:44559
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:38257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:44559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:34.918226 24817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:34.918608 24817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:34.918748 24817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:34.926270 24817 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:34.926402 24817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:34.926456 24817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:34.926528 24817 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:34.926633 24817 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:34.937047 24817 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:38257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:44559
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:44559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:34.938414 24817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:34.939833 24817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:34.952786 24825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:34.952822 24822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:34.952857 24823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:34.953727 24817 server_base.cc:1034] running on GCE node
I20260620 23:04:34.954542 24817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:34.956068 24817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:34.957283 24817 hybrid_clock.cc:648] HybridClock initialized: now 1781996674957238 us; error 38 us; skew 500 ppm
I20260620 23:04:34.957680 24817 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:04:34 dist-test-slave-f63b krb5kdc[24804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996674, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:34.961135 24817 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 23:04:34.962723 24817 webserver.cc:460] Webserver started at http://127.2.237.62:34311/ using document root <none> and password file <none>
I20260620 23:04:34.963375 24817 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:34.963474 24817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:34.963721 24817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:34.966686 24817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "ee48bb362aa74238b8acc48bfdce8c27"
format_stamp: "Formatted at 2026-06-20 23:04:34 on dist-test-slave-f63b"
server_key: "521931d5109032041b56cca682c94429"
server_key_iv: "61da9ec60aad385d11e4334250c9c695"
server_key_version: "encryptionkey@0"
I20260620 23:04:34.967411 24817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "ee48bb362aa74238b8acc48bfdce8c27"
format_stamp: "Formatted at 2026-06-20 23:04:34 on dist-test-slave-f63b"
server_key: "521931d5109032041b56cca682c94429"
server_key_iv: "61da9ec60aad385d11e4334250c9c695"
server_key_version: "encryptionkey@0"
I20260620 23:04:34.972751 24817 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260620 23:04:34.976483 24832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:34.977895 24817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260620 23:04:34.978094 24817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "ee48bb362aa74238b8acc48bfdce8c27"
format_stamp: "Formatted at 2026-06-20 23:04:34 on dist-test-slave-f63b"
server_key: "521931d5109032041b56cca682c94429"
server_key_iv: "61da9ec60aad385d11e4334250c9c695"
server_key_version: "encryptionkey@0"
I20260620 23:04:34.978298 24817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:35.002754 24817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:35.006968 24817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:35.007267 24817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:35.024997 24817 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:44559
I20260620 23:04:35.025022 24883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:44559 every 8 connection(s)
I20260620 23:04:35.026433 24817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:04:35.030387 24884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:35.031167 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 24817
I20260620 23:04:35.031391 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:04:35.031915 2996 external_mini_cluster.cc:1428] Setting key 78331bff3aba182e317ce68ca8e36e03
I20260620 23:04:35.045068 24884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee48bb362aa74238b8acc48bfdce8c27: Bootstrap starting.
I20260620 23:04:35.047171 24887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:35.033945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35863 (local address 127.2.237.62:44559)
0620 23:04:35.034824 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:35.034845 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:35.034897 (+ 52us) server_negotiation.cc:407] Connection header received
0620 23:04:35.037052 (+ 2155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:35.037082 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:35.037638 (+ 556us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:35.038102 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:35.038654 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:35.040556 (+ 1902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:35.042085 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:35.043200 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:35.043370 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:35.044683 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:35.044690 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:35.044708 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:35.045318 (+ 610us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:35.045420 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:35.045445 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:35.045564 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:35.045967 (+ 403us) server_negotiation.cc:299] Negotiation successful
0620 23:04:35.046155 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":350,"threads_started":1}
I20260620 23:04:35.048550 24884 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee48bb362aa74238b8acc48bfdce8c27: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:35.049898 24884 log.cc:824] T 00000000000000000000000000000000 P ee48bb362aa74238b8acc48bfdce8c27: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:35.051420 2996 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:04:35.052913 24884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee48bb362aa74238b8acc48bfdce8c27: No bootstrap required, opened a new log
I20260620 23:04:35.062593 24887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:35.055635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32911 (local address 127.2.237.62:44559)
0620 23:04:35.055820 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:35.055826 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:35.056330 (+ 504us) server_negotiation.cc:407] Connection header received
0620 23:04:35.056482 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:35.056489 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:35.056581 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:35.056708 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:35.057172 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:35.058393 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:35.059725 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:35.060477 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:35.060618 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:35.061664 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:35.061670 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:35.061674 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:35.061694 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:35.061762 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:35.061781 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:35.061920 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:35.062125 (+ 205us) server_negotiation.cc:299] Negotiation successful
0620 23:04:35.062177 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 23:04:35.062252 24884 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee48bb362aa74238b8acc48bfdce8c27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee48bb362aa74238b8acc48bfdce8c27" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44559 } }
I20260620 23:04:35.062785 24884 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee48bb362aa74238b8acc48bfdce8c27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:35.062932 24884 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee48bb362aa74238b8acc48bfdce8c27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee48bb362aa74238b8acc48bfdce8c27, State: Initialized, Role: FOLLOWER
I20260620 23:04:35.063616 24884 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee48bb362aa74238b8acc48bfdce8c27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee48bb362aa74238b8acc48bfdce8c27" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44559 } }
I20260620 23:04:35.063881 24884 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee48bb362aa74238b8acc48bfdce8c27 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:04:35.064056 24884 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee48bb362aa74238b8acc48bfdce8c27 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:04:35.064245 24884 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee48bb362aa74238b8acc48bfdce8c27 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:35.066432 24884 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee48bb362aa74238b8acc48bfdce8c27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee48bb362aa74238b8acc48bfdce8c27" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44559 } }
I20260620 23:04:35.066959 24884 leader_election.cc:302] T 00000000000000000000000000000000 P ee48bb362aa74238b8acc48bfdce8c27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee48bb362aa74238b8acc48bfdce8c27; no voters:
I20260620 23:04:35.067569 24884 leader_election.cc:288] T 00000000000000000000000000000000 P ee48bb362aa74238b8acc48bfdce8c27 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:04:35.067669 24894 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee48bb362aa74238b8acc48bfdce8c27 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:35.068462 24894 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee48bb362aa74238b8acc48bfdce8c27 [term 1 LEADER]: Becoming Leader. State: Replica: ee48bb362aa74238b8acc48bfdce8c27, State: Running, Role: LEADER
I20260620 23:04:35.069010 24894 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee48bb362aa74238b8acc48bfdce8c27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee48bb362aa74238b8acc48bfdce8c27" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44559 } }
I20260620 23:04:35.069708 24884 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee48bb362aa74238b8acc48bfdce8c27 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:35.072202 24896 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee48bb362aa74238b8acc48bfdce8c27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee48bb362aa74238b8acc48bfdce8c27. Latest consensus state: current_term: 1 leader_uuid: "ee48bb362aa74238b8acc48bfdce8c27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee48bb362aa74238b8acc48bfdce8c27" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44559 } } }
I20260620 23:04:35.072674 24896 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee48bb362aa74238b8acc48bfdce8c27 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:35.073100 24895 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee48bb362aa74238b8acc48bfdce8c27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee48bb362aa74238b8acc48bfdce8c27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee48bb362aa74238b8acc48bfdce8c27" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44559 } } }
I20260620 23:04:35.073532 24895 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee48bb362aa74238b8acc48bfdce8c27 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:35.080255 24902 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:04:35.087777 24902 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:04:35.096611 24902 catalog_manager.cc:1347] Generated new cluster ID: f7604137c7c4427bb8538e5e401bad5e
I20260620 23:04:35.096753 24902 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:04:35.110795 24902 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:04:35.112203 24902 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:04:35.124555 24902 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee48bb362aa74238b8acc48bfdce8c27: Generated new TSK 0
I20260620 23:04:35.125501 24902 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:04:35.140061 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 24817
W20260620 23:04:35.141305 24879 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35863
W20260620 23:04:35.142097 2996 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 24817: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260620 23:04:35.142252 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 24817
W20260620 23:04:35.160176 24802 connection.cc:535] client connection to 127.2.237.62:44559 recv error: Network error: failed to read from TLS socket (remote: 127.2.237.62:44559): Connection reset by peer (error 104)
2026-06-20T23:04:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1950 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:04:35 dist-test-slave-f63b krb5kdc[24915](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:04:35 dist-test-slave-f63b krb5kdc[24915](info): set up 2 sockets
Jun 20 23:04:35 dist-test-slave-f63b krb5kdc[24915](info): commencing operation
krb5kdc: starting...
W20260620 23:04:36.247323 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.005s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 23:04:36 dist-test-slave-f63b krb5kdc[24915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:04:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:04:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
I20260620 23:04:36.508026 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:34451
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:45157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:34451
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:36.846380 24928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:36.846693 24928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:36.846942 24928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:36.854344 24928 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:36.854477 24928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:36.854612 24928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:36.854720 24928 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:36.854799 24928 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:36.865168 24928 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:45157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:34451
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:34451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:36.866624 24928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:36.868026 24928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:36.880038 24933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:36.880124 24934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:36.880088 24936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:36.881093 24928 server_base.cc:1034] running on GCE node
I20260620 23:04:36.881999 24928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:36.883616 24928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:36.884828 24928 hybrid_clock.cc:648] HybridClock initialized: now 1781996676884786 us; error 31 us; skew 500 ppm
I20260620 23:04:36.885231 24928 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:04:36 dist-test-slave-f63b krb5kdc[24915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996676, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:36.888504 24928 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 23:04:36.890021 24928 webserver.cc:460] Webserver started at http://127.2.237.62:40117/ using document root <none> and password file <none>
I20260620 23:04:36.890806 24928 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:36.890959 24928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:36.891279 24928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:36.894395 24928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "fcc65c17eb29469f86f1fc50630da024"
format_stamp: "Formatted at 2026-06-20 23:04:36 on dist-test-slave-f63b"
server_key: "d8bdfc9a7f10d2a532df4edc8e03f388"
server_key_iv: "edcc25db56d41405c22c140332c64330"
server_key_version: "encryptionkey@0"
I20260620 23:04:36.895237 24928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "fcc65c17eb29469f86f1fc50630da024"
format_stamp: "Formatted at 2026-06-20 23:04:36 on dist-test-slave-f63b"
server_key: "d8bdfc9a7f10d2a532df4edc8e03f388"
server_key_iv: "edcc25db56d41405c22c140332c64330"
server_key_version: "encryptionkey@0"
I20260620 23:04:36.900480 24928 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260620 23:04:36.903921 24943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:36.905241 24928 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 23:04:36.905431 24928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "fcc65c17eb29469f86f1fc50630da024"
format_stamp: "Formatted at 2026-06-20 23:04:36 on dist-test-slave-f63b"
server_key: "d8bdfc9a7f10d2a532df4edc8e03f388"
server_key_iv: "edcc25db56d41405c22c140332c64330"
server_key_version: "encryptionkey@0"
I20260620 23:04:36.905638 24928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:36.911448 24928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:36.915964 24928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:36.916297 24928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:36.935452 24928 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:34451
I20260620 23:04:36.935539 24994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:34451 every 8 connection(s)
I20260620 23:04:36.936937 24928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:04:36.940831 24995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:36.944378 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 24928
I20260620 23:04:36.944557 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:04:36.945124 2996 external_mini_cluster.cc:1428] Setting key f297d6b0553af88f18f564f6a429d9a2
I20260620 23:04:36.953253 24998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:36.947137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51149 (local address 127.2.237.62:34451)
0620 23:04:36.947953 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:36.947965 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:36.948006 (+ 41us) server_negotiation.cc:407] Connection header received
0620 23:04:36.949798 (+ 1792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:36.949821 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:36.950205 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:36.950544 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:36.951656 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:36.951662 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:36.951675 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:36.951698 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:36.951756 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:36.951777 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:36.952000 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:36.952098 (+ 98us) server_negotiation.cc:299] Negotiation successful
0620 23:04:36.952317 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":378,"threads_started":1}
I20260620 23:04:36.954042 24995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcc65c17eb29469f86f1fc50630da024: Bootstrap starting.
I20260620 23:04:36.957267 2996 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:04:36.957454 24995 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcc65c17eb29469f86f1fc50630da024: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:36.958773 24995 log.cc:824] T 00000000000000000000000000000000 P fcc65c17eb29469f86f1fc50630da024: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:36.962319 24995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcc65c17eb29469f86f1fc50630da024: No bootstrap required, opened a new log
I20260620 23:04:36.965108 24998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:36.962533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45661 (local address 127.2.237.62:34451)
0620 23:04:36.962728 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:36.962736 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:36.963242 (+ 506us) server_negotiation.cc:407] Connection header received
0620 23:04:36.963361 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:36.963366 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:36.963432 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:36.963543 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:36.964437 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:36.964448 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:36.964453 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:36.964474 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:36.964531 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:36.964557 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:36.964679 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:36.964751 (+ 72us) server_negotiation.cc:299] Negotiation successful
0620 23:04:36.964811 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260620 23:04:36.971566 24995 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcc65c17eb29469f86f1fc50630da024 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcc65c17eb29469f86f1fc50630da024" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 34451 } }
I20260620 23:04:36.971944 24995 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcc65c17eb29469f86f1fc50630da024 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:36.972018 24995 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcc65c17eb29469f86f1fc50630da024 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcc65c17eb29469f86f1fc50630da024, State: Initialized, Role: FOLLOWER
I20260620 23:04:36.972549 24995 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcc65c17eb29469f86f1fc50630da024 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcc65c17eb29469f86f1fc50630da024" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 34451 } }
I20260620 23:04:36.972755 24995 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcc65c17eb29469f86f1fc50630da024 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:04:36.972851 24995 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcc65c17eb29469f86f1fc50630da024 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:04:36.972982 24995 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcc65c17eb29469f86f1fc50630da024 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:36.975338 24995 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcc65c17eb29469f86f1fc50630da024 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcc65c17eb29469f86f1fc50630da024" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 34451 } }
I20260620 23:04:36.975994 24995 leader_election.cc:302] T 00000000000000000000000000000000 P fcc65c17eb29469f86f1fc50630da024 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcc65c17eb29469f86f1fc50630da024; no voters:
I20260620 23:04:36.976649 24995 leader_election.cc:288] T 00000000000000000000000000000000 P fcc65c17eb29469f86f1fc50630da024 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:04:36.976794 25005 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcc65c17eb29469f86f1fc50630da024 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:36.977444 25005 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcc65c17eb29469f86f1fc50630da024 [term 1 LEADER]: Becoming Leader. State: Replica: fcc65c17eb29469f86f1fc50630da024, State: Running, Role: LEADER
I20260620 23:04:36.978026 25005 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcc65c17eb29469f86f1fc50630da024 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcc65c17eb29469f86f1fc50630da024" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 34451 } }
I20260620 23:04:36.978878 24995 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcc65c17eb29469f86f1fc50630da024 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:36.981289 25006 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcc65c17eb29469f86f1fc50630da024 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fcc65c17eb29469f86f1fc50630da024" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcc65c17eb29469f86f1fc50630da024" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 34451 } } }
I20260620 23:04:36.981364 25007 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcc65c17eb29469f86f1fc50630da024 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcc65c17eb29469f86f1fc50630da024. Latest consensus state: current_term: 1 leader_uuid: "fcc65c17eb29469f86f1fc50630da024" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcc65c17eb29469f86f1fc50630da024" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 34451 } } }
I20260620 23:04:36.981709 25006 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcc65c17eb29469f86f1fc50630da024 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:36.981832 25007 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcc65c17eb29469f86f1fc50630da024 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:36.982964 25013 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:04:36.989097 25013 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:04:36.998365 25013 catalog_manager.cc:1347] Generated new cluster ID: 3c0091ff14b745a0bdccad0d16f0cf53
I20260620 23:04:36.998536 25013 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:04:37.018687 25013 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:04:37.019250 25013 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:04:37.030215 25013 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fcc65c17eb29469f86f1fc50630da024: Generated new TSK 0
I20260620 23:04:37.031289 25013 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:04:37.042057 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 24928
W20260620 23:04:37.043421 24990 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51149
W20260620 23:04:37.044124 2996 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 24928: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260620 23:04:37.044291 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 24928
2026-06-20T23:04:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1900 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:04:37 dist-test-slave-f63b krb5kdc[25026](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:04:37 dist-test-slave-f63b krb5kdc[25026](info): set up 2 sockets
Jun 20 23:04:37 dist-test-slave-f63b krb5kdc[25026](info): commencing operation
krb5kdc: starting...
W20260620 23:04:38.151098 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.003s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 23:04:38 dist-test-slave-f63b krb5kdc[25026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:04:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:04:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
I20260620 23:04:38.406294 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:44005
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:44221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:44005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:38.752112 25039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:38.752463 25039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:38.752631 25039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:38.760130 25039 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:38.760279 25039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:38.760344 25039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:38.760399 25039 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:38.760443 25039 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:38.770690 25039 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:44221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:44005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:44005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:38.772022 25039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:38.773290 25039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:38.785470 25044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:38.785394 25047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:38.785506 25045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:38.786355 25039 server_base.cc:1034] running on GCE node
I20260620 23:04:38.787240 25039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:38.788604 25039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:38.789800 25039 hybrid_clock.cc:648] HybridClock initialized: now 1781996678789746 us; error 52 us; skew 500 ppm
I20260620 23:04:38.790118 25039 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:04:38 dist-test-slave-f63b krb5kdc[25026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996678, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:38.798331 25039 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 23:04:38.800004 25039 webserver.cc:460] Webserver started at http://127.2.237.62:35631/ using document root <none> and password file <none>
I20260620 23:04:38.800738 25039 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:38.800870 25039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:38.801204 25039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:38.804329 25039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "11a61aa725f5439fbee0b6e63305978a"
format_stamp: "Formatted at 2026-06-20 23:04:38 on dist-test-slave-f63b"
server_key: "3b6425fe5c0d045ce6119c4229b7fdeb"
server_key_iv: "3e28ce7a8b16417c36a45d4ac1653ba8"
server_key_version: "encryptionkey@0"
I20260620 23:04:38.805126 25039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "11a61aa725f5439fbee0b6e63305978a"
format_stamp: "Formatted at 2026-06-20 23:04:38 on dist-test-slave-f63b"
server_key: "3b6425fe5c0d045ce6119c4229b7fdeb"
server_key_iv: "3e28ce7a8b16417c36a45d4ac1653ba8"
server_key_version: "encryptionkey@0"
I20260620 23:04:38.810505 25039 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260620 23:04:38.814080 25054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:38.815445 25039 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260620 23:04:38.815655 25039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "11a61aa725f5439fbee0b6e63305978a"
format_stamp: "Formatted at 2026-06-20 23:04:38 on dist-test-slave-f63b"
server_key: "3b6425fe5c0d045ce6119c4229b7fdeb"
server_key_iv: "3e28ce7a8b16417c36a45d4ac1653ba8"
server_key_version: "encryptionkey@0"
I20260620 23:04:38.815871 25039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:38.833200 25039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:38.837409 25039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:38.837723 25039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:38.857075 25039 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:44005
I20260620 23:04:38.857122 25105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:44005 every 8 connection(s)
I20260620 23:04:38.858493 25039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:04:38.862226 25106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:38.862332 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 25039
I20260620 23:04:38.862506 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:04:38.862990 2996 external_mini_cluster.cc:1428] Setting key 114e0fd476272e76cc3bb668039dd7c1
I20260620 23:04:38.876816 25106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11a61aa725f5439fbee0b6e63305978a: Bootstrap starting.
Jun 20 23:04:38 dist-test-slave-f63b krb5kdc[25026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:04:38.880722 25106 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11a61aa725f5439fbee0b6e63305978a: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:38.882089 25106 log.cc:824] T 00000000000000000000000000000000 P 11a61aa725f5439fbee0b6e63305978a: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:38.885600 25106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11a61aa725f5439fbee0b6e63305978a: No bootstrap required, opened a new log
I20260620 23:04:38.887311 25109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:38.864868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55875 (local address 127.2.237.62:44005)
0620 23:04:38.865713 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:38.865729 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:38.865789 (+ 60us) server_negotiation.cc:407] Connection header received
0620 23:04:38.867596 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:38.867638 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:38.868187 (+ 549us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:38.868647 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:38.869959 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:38.871856 (+ 1897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:38.873110 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:38.874227 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:38.874438 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:38.878479 (+ 4041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:38.878489 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:38.878535 (+ 46us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:38.878625 (+ 90us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:38.882643 (+ 4018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:38.883566 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:38.883574 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:38.883582 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:38.883780 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:38.884152 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:38.884157 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:38.884159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:38.884851 (+ 692us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:38.885164 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:38.885489 (+ 325us) server_negotiation.cc:299] Negotiation successful
0620 23:04:38.885932 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":348,"threads_started":1}
I20260620 23:04:38.891201 2996 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:04:38.894347 25106 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11a61aa725f5439fbee0b6e63305978a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a61aa725f5439fbee0b6e63305978a" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44005 } }
I20260620 23:04:38.894800 25106 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11a61aa725f5439fbee0b6e63305978a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:38.894906 25106 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11a61aa725f5439fbee0b6e63305978a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11a61aa725f5439fbee0b6e63305978a, State: Initialized, Role: FOLLOWER
I20260620 23:04:38.895609 25106 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11a61aa725f5439fbee0b6e63305978a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a61aa725f5439fbee0b6e63305978a" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44005 } }
I20260620 23:04:38.895917 25106 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11a61aa725f5439fbee0b6e63305978a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:04:38.896075 25106 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11a61aa725f5439fbee0b6e63305978a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:04:38.896248 25106 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11a61aa725f5439fbee0b6e63305978a [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:38.899353 25106 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11a61aa725f5439fbee0b6e63305978a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a61aa725f5439fbee0b6e63305978a" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44005 } }
I20260620 23:04:38.899964 25106 leader_election.cc:302] T 00000000000000000000000000000000 P 11a61aa725f5439fbee0b6e63305978a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11a61aa725f5439fbee0b6e63305978a; no voters:
I20260620 23:04:38.900604 25106 leader_election.cc:288] T 00000000000000000000000000000000 P 11a61aa725f5439fbee0b6e63305978a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:04:38.900719 25116 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11a61aa725f5439fbee0b6e63305978a [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:38.901366 25116 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11a61aa725f5439fbee0b6e63305978a [term 1 LEADER]: Becoming Leader. State: Replica: 11a61aa725f5439fbee0b6e63305978a, State: Running, Role: LEADER
I20260620 23:04:38.901964 25116 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11a61aa725f5439fbee0b6e63305978a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a61aa725f5439fbee0b6e63305978a" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44005 } }
I20260620 23:04:38.904650 25106 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11a61aa725f5439fbee0b6e63305978a [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:38.904925 25118 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11a61aa725f5439fbee0b6e63305978a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11a61aa725f5439fbee0b6e63305978a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a61aa725f5439fbee0b6e63305978a" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44005 } } }
I20260620 23:04:38.905311 25118 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11a61aa725f5439fbee0b6e63305978a [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:38.905865 25117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11a61aa725f5439fbee0b6e63305978a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11a61aa725f5439fbee0b6e63305978a. Latest consensus state: current_term: 1 leader_uuid: "11a61aa725f5439fbee0b6e63305978a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a61aa725f5439fbee0b6e63305978a" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 44005 } } }
I20260620 23:04:38.906265 25117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11a61aa725f5439fbee0b6e63305978a [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:38.916965 25120 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:04:38.919124 25109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:38.895684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36245 (local address 127.2.237.62:44005)
0620 23:04:38.895924 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:38.895934 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:38.896411 (+ 477us) server_negotiation.cc:407] Connection header received
0620 23:04:38.896666 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:38.896670 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:38.896731 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:38.896810 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:38.898675 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:38.899368 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:38.901254 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:38.902253 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:38.902379 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:38.905759 (+ 3380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:38.905767 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:38.905772 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:38.905808 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:38.910884 (+ 5076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:38.915355 (+ 4471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:38.915363 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:38.915369 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:38.915554 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:38.916584 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:38.916591 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:38.916597 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:38.917204 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:38.917544 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:38.918282 (+ 738us) server_negotiation.cc:299] Negotiation successful
0620 23:04:38.918540 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260620 23:04:38.926385 25120 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:04:38.937529 25120 catalog_manager.cc:1347] Generated new cluster ID: abc9f50edb4d428cbc06fabee69da48d
I20260620 23:04:38.937649 25120 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:04:38.963083 25120 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:04:38.964504 25120 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:04:38.972182 25120 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11a61aa725f5439fbee0b6e63305978a: Generated new TSK 0
I20260620 23:04:38.973080 25120 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:04:38.980940 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 25039
I20260620 23:04:38.982077 25101 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:04:39.076798 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 25039
2026-06-20T23:04:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2033 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:04:39 dist-test-slave-f63b krb5kdc[25139](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:04:39 dist-test-slave-f63b krb5kdc[25139](info): set up 2 sockets
Jun 20 23:04:39 dist-test-slave-f63b krb5kdc[25139](info): commencing operation
krb5kdc: starting...
W20260620 23:04:40.250989 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.003s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 23:04:40 dist-test-slave-f63b krb5kdc[25139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:04:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:04:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
I20260620 23:04:40.505023 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:43857
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:39563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:43857
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:40.841305 25152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:40.841603 25152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:40.841696 25152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:40.849030 25152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:40.849155 25152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:40.849202 25152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:40.849265 25152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:40.849309 25152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:40.859289 25152 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:39563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:43857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:43857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:40.860711 25152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:40.861979 25152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:40.874102 25157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:40.874125 25158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:40.874056 25160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:40.875303 25152 server_base.cc:1034] running on GCE node
I20260620 23:04:40.876166 25152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:40.877677 25152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:40.878937 25152 hybrid_clock.cc:648] HybridClock initialized: now 1781996680878886 us; error 45 us; skew 500 ppm
I20260620 23:04:40.879300 25152 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:04:40 dist-test-slave-f63b krb5kdc[25139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996680, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:40.882498 25152 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 23:04:40.884140 25152 webserver.cc:460] Webserver started at http://127.2.237.62:43311/ using document root <none> and password file <none>
I20260620 23:04:40.884868 25152 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:40.885020 25152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:40.885334 25152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:40.888507 25152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "1e2bceda95154e858b495c000e551775"
format_stamp: "Formatted at 2026-06-20 23:04:40 on dist-test-slave-f63b"
server_key: "63fded0ded7a37670d5973cfb029cc26"
server_key_iv: "bc744ce368301480ab7341e7aa72ba1c"
server_key_version: "encryptionkey@0"
I20260620 23:04:40.889312 25152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "1e2bceda95154e858b495c000e551775"
format_stamp: "Formatted at 2026-06-20 23:04:40 on dist-test-slave-f63b"
server_key: "63fded0ded7a37670d5973cfb029cc26"
server_key_iv: "bc744ce368301480ab7341e7aa72ba1c"
server_key_version: "encryptionkey@0"
I20260620 23:04:40.895058 25152 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260620 23:04:40.898921 25167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:40.900424 25152 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 23:04:40.900628 25152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "1e2bceda95154e858b495c000e551775"
format_stamp: "Formatted at 2026-06-20 23:04:40 on dist-test-slave-f63b"
server_key: "63fded0ded7a37670d5973cfb029cc26"
server_key_iv: "bc744ce368301480ab7341e7aa72ba1c"
server_key_version: "encryptionkey@0"
I20260620 23:04:40.900843 25152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:40.912034 25152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:40.916674 25152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:40.917006 25152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:40.936482 25152 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:43857
I20260620 23:04:40.936513 25218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:43857 every 8 connection(s)
I20260620 23:04:40.937953 25152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:04:40.940759 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 25152
I20260620 23:04:40.940937 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:04:40.941463 2996 external_mini_cluster.cc:1428] Setting key 49d7c727c7501d4d277359e59a03e60c
I20260620 23:04:40.942543 25219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:40.955320 25222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:40.943259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47091 (local address 127.2.237.62:43857)
0620 23:04:40.944026 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:40.944038 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:40.944079 (+ 41us) server_negotiation.cc:407] Connection header received
0620 23:04:40.945637 (+ 1558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:40.945660 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:40.946071 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:40.946427 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:40.947053 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:40.948422 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:40.950303 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:40.951113 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:40.951251 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:40.952634 (+ 1383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:40.952643 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:40.952661 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:40.953312 (+ 651us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:40.953412 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:40.953440 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:40.953547 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:40.953795 (+ 248us) server_negotiation.cc:299] Negotiation successful
0620 23:04:40.954186 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":358,"threads_started":1}
I20260620 23:04:40.956473 25219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e2bceda95154e858b495c000e551775: Bootstrap starting.
I20260620 23:04:40.959427 2996 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:04:40.960013 25219 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e2bceda95154e858b495c000e551775: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:40.961258 25219 log.cc:824] T 00000000000000000000000000000000 P 1e2bceda95154e858b495c000e551775: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:40.964499 25219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e2bceda95154e858b495c000e551775: No bootstrap required, opened a new log
I20260620 23:04:40.971136 25222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:40.964756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32799 (local address 127.2.237.62:43857)
0620 23:04:40.964983 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:40.964989 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:40.965321 (+ 332us) server_negotiation.cc:407] Connection header received
0620 23:04:40.965570 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:40.965575 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:40.965636 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:40.965760 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:40.966303 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:40.967169 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:40.968530 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:40.969020 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:40.969206 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:40.970322 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:40.970328 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:40.970331 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:40.970349 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:40.970386 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:40.970401 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:40.970510 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:40.970708 (+ 198us) server_negotiation.cc:299] Negotiation successful
0620 23:04:40.970819 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260620 23:04:40.973289 25219 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e2bceda95154e858b495c000e551775 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e2bceda95154e858b495c000e551775" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43857 } }
I20260620 23:04:40.973702 25219 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e2bceda95154e858b495c000e551775 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:40.973822 25219 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e2bceda95154e858b495c000e551775 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e2bceda95154e858b495c000e551775, State: Initialized, Role: FOLLOWER
I20260620 23:04:40.974556 25219 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e2bceda95154e858b495c000e551775 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e2bceda95154e858b495c000e551775" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43857 } }
I20260620 23:04:40.974861 25219 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e2bceda95154e858b495c000e551775 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:04:40.975054 25219 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e2bceda95154e858b495c000e551775 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:04:40.975253 25219 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e2bceda95154e858b495c000e551775 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:40.977433 25219 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e2bceda95154e858b495c000e551775 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e2bceda95154e858b495c000e551775" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43857 } }
I20260620 23:04:40.977960 25219 leader_election.cc:302] T 00000000000000000000000000000000 P 1e2bceda95154e858b495c000e551775 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e2bceda95154e858b495c000e551775; no voters:
I20260620 23:04:40.978552 25219 leader_election.cc:288] T 00000000000000000000000000000000 P 1e2bceda95154e858b495c000e551775 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:04:40.978754 25229 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e2bceda95154e858b495c000e551775 [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:40.979485 25229 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e2bceda95154e858b495c000e551775 [term 1 LEADER]: Becoming Leader. State: Replica: 1e2bceda95154e858b495c000e551775, State: Running, Role: LEADER
I20260620 23:04:40.979984 25229 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e2bceda95154e858b495c000e551775 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e2bceda95154e858b495c000e551775" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43857 } }
I20260620 23:04:40.980816 25219 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e2bceda95154e858b495c000e551775 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:40.983404 25230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e2bceda95154e858b495c000e551775 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e2bceda95154e858b495c000e551775" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e2bceda95154e858b495c000e551775" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43857 } } }
I20260620 23:04:40.983492 25231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e2bceda95154e858b495c000e551775 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e2bceda95154e858b495c000e551775. Latest consensus state: current_term: 1 leader_uuid: "1e2bceda95154e858b495c000e551775" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e2bceda95154e858b495c000e551775" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43857 } } }
I20260620 23:04:40.983815 25230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e2bceda95154e858b495c000e551775 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:40.983947 25231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e2bceda95154e858b495c000e551775 [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:40.985519 25236 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:04:40.993280 25236 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:04:41.001927 25236 catalog_manager.cc:1347] Generated new cluster ID: 1f60e18944a6413cbb8bd6db6c7c4758
I20260620 23:04:41.002055 25236 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:04:41.044976 25236 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:04:41.047030 25236 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:04:41.058362 25236 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e2bceda95154e858b495c000e551775: Generated new TSK 0
I20260620 23:04:41.059471 25236 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:04:41.078105 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 25152
W20260620 23:04:41.079325 25214 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47091
W20260620 23:04:41.080060 2996 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 25152: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260620 23:04:41.080210 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 25152
2026-06-20T23:04:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2002 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:04:41 dist-test-slave-f63b krb5kdc[25250](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:04:41 dist-test-slave-f63b krb5kdc[25250](info): set up 2 sockets
Jun 20 23:04:41 dist-test-slave-f63b krb5kdc[25250](info): commencing operation
krb5kdc: starting...
W20260620 23:04:42.199222 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.007s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 23:04:42 dist-test-slave-f63b krb5kdc[25250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:04:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:04:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
I20260620 23:04:42.455973 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:45905
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:46839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:45905
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:42.792562 25263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:42.792863 25263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:42.793008 25263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:42.800545 25263 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:42.800662 25263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:42.800789 25263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:42.800895 25263 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:42.800973 25263 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:42.810959 25263 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:46839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:45905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:45905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:42.812278 25263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:42.813563 25263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:42.825525 25268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:42.825443 25271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:42.825536 25269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:42.826181 25263 server_base.cc:1034] running on GCE node
I20260620 23:04:42.827052 25263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:42.828509 25263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:42.829748 25263 hybrid_clock.cc:648] HybridClock initialized: now 1781996682829699 us; error 42 us; skew 500 ppm
I20260620 23:04:42.830107 25263 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:04:42 dist-test-slave-f63b krb5kdc[25250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996682, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:42.833287 25263 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 23:04:42.834950 25263 webserver.cc:460] Webserver started at http://127.2.237.62:34411/ using document root <none> and password file <none>
I20260620 23:04:42.835713 25263 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:42.835872 25263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:42.836203 25263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:42.839570 25263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "228559f4c0ad4c4f97d88e976da36ede"
format_stamp: "Formatted at 2026-06-20 23:04:42 on dist-test-slave-f63b"
server_key: "ecda343ec93c034bdda2d04c152e31eb"
server_key_iv: "156973356a7c195a2be0d9b151fb6d7f"
server_key_version: "encryptionkey@0"
I20260620 23:04:42.840392 25263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "228559f4c0ad4c4f97d88e976da36ede"
format_stamp: "Formatted at 2026-06-20 23:04:42 on dist-test-slave-f63b"
server_key: "ecda343ec93c034bdda2d04c152e31eb"
server_key_iv: "156973356a7c195a2be0d9b151fb6d7f"
server_key_version: "encryptionkey@0"
I20260620 23:04:42.845896 25263 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260620 23:04:42.849691 25278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:42.851246 25263 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260620 23:04:42.851445 25263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "228559f4c0ad4c4f97d88e976da36ede"
format_stamp: "Formatted at 2026-06-20 23:04:42 on dist-test-slave-f63b"
server_key: "ecda343ec93c034bdda2d04c152e31eb"
server_key_iv: "156973356a7c195a2be0d9b151fb6d7f"
server_key_version: "encryptionkey@0"
I20260620 23:04:42.851658 25263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:42.857329 25263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:42.861714 25263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:42.862031 25263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:42.880802 25263 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:45905
I20260620 23:04:42.880966 25329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:45905 every 8 connection(s)
I20260620 23:04:42.882262 25263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:04:42.886214 25330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:42.891248 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 25263
I20260620 23:04:42.891427 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:04:42.891875 2996 external_mini_cluster.cc:1428] Setting key c6f01e14e3162961f788fa663f041bc1
I20260620 23:04:42.897765 25330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 228559f4c0ad4c4f97d88e976da36ede: Bootstrap starting.
I20260620 23:04:42.899891 25333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:42.893720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58571 (local address 127.2.237.62:45905)
0620 23:04:42.894563 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:42.894623 (+ 60us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:42.894677 (+ 54us) server_negotiation.cc:407] Connection header received
0620 23:04:42.896450 (+ 1773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:42.896474 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:42.896833 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:42.897173 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:42.898250 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:42.898256 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:42.898270 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:42.898289 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:42.898345 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:42.898367 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:42.898546 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:42.898711 (+ 165us) server_negotiation.cc:299] Negotiation successful
0620 23:04:42.898924 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":414,"threads_started":1}
I20260620 23:04:42.901396 25330 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 228559f4c0ad4c4f97d88e976da36ede: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:42.902743 25330 log.cc:824] T 00000000000000000000000000000000 P 228559f4c0ad4c4f97d88e976da36ede: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:42.903970 2996 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:04:42.906038 25330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 228559f4c0ad4c4f97d88e976da36ede: No bootstrap required, opened a new log
I20260620 23:04:42.912068 25333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:42.909052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33013 (local address 127.2.237.62:45905)
0620 23:04:42.909258 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:42.909268 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:42.909870 (+ 602us) server_negotiation.cc:407] Connection header received
0620 23:04:42.910059 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:42.910064 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:42.910134 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:42.910251 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:42.911375 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:42.911381 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:42.911387 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0620 23:04:42.911406 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:42.911466 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0620 23:04:42.911486 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:42.911615 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:42.911696 (+ 81us) server_negotiation.cc:299] Negotiation successful
0620 23:04:42.911759 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260620 23:04:42.916026 25330 raft_consensus.cc:348] T 00000000000000000000000000000000 P 228559f4c0ad4c4f97d88e976da36ede [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "228559f4c0ad4c4f97d88e976da36ede" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 45905 } }
I20260620 23:04:42.916402 25330 raft_consensus.cc:374] T 00000000000000000000000000000000 P 228559f4c0ad4c4f97d88e976da36ede [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260620 23:04:42.916486 25330 raft_consensus.cc:729] T 00000000000000000000000000000000 P 228559f4c0ad4c4f97d88e976da36ede [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 228559f4c0ad4c4f97d88e976da36ede, State: Initialized, Role: FOLLOWER
I20260620 23:04:42.917074 25330 consensus_queue.cc:260] T 00000000000000000000000000000000 P 228559f4c0ad4c4f97d88e976da36ede [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "228559f4c0ad4c4f97d88e976da36ede" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 45905 } }
I20260620 23:04:42.917291 25330 raft_consensus.cc:388] T 00000000000000000000000000000000 P 228559f4c0ad4c4f97d88e976da36ede [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260620 23:04:42.917383 25330 raft_consensus.cc:482] T 00000000000000000000000000000000 P 228559f4c0ad4c4f97d88e976da36ede [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260620 23:04:42.917524 25330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 228559f4c0ad4c4f97d88e976da36ede [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:42.919816 25330 raft_consensus.cc:504] T 00000000000000000000000000000000 P 228559f4c0ad4c4f97d88e976da36ede [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "228559f4c0ad4c4f97d88e976da36ede" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 45905 } }
I20260620 23:04:42.920269 25330 leader_election.cc:302] T 00000000000000000000000000000000 P 228559f4c0ad4c4f97d88e976da36ede [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 228559f4c0ad4c4f97d88e976da36ede; no voters:
I20260620 23:04:42.920850 25330 leader_election.cc:288] T 00000000000000000000000000000000 P 228559f4c0ad4c4f97d88e976da36ede [CANDIDATE]: Term 1 election: Requested vote from peers
I20260620 23:04:42.921064 25340 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 228559f4c0ad4c4f97d88e976da36ede [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:42.921746 25340 raft_consensus.cc:686] T 00000000000000000000000000000000 P 228559f4c0ad4c4f97d88e976da36ede [term 1 LEADER]: Becoming Leader. State: Replica: 228559f4c0ad4c4f97d88e976da36ede, State: Running, Role: LEADER
I20260620 23:04:42.922407 25340 consensus_queue.cc:237] T 00000000000000000000000000000000 P 228559f4c0ad4c4f97d88e976da36ede [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "228559f4c0ad4c4f97d88e976da36ede" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 45905 } }
I20260620 23:04:42.923131 25330 sys_catalog.cc:564] T 00000000000000000000000000000000 P 228559f4c0ad4c4f97d88e976da36ede [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:42.925446 25341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 228559f4c0ad4c4f97d88e976da36ede [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "228559f4c0ad4c4f97d88e976da36ede" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "228559f4c0ad4c4f97d88e976da36ede" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 45905 } } }
I20260620 23:04:42.925935 25341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 228559f4c0ad4c4f97d88e976da36ede [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:42.926522 25342 sys_catalog.cc:455] T 00000000000000000000000000000000 P 228559f4c0ad4c4f97d88e976da36ede [sys.catalog]: SysCatalogTable state changed. Reason: New leader 228559f4c0ad4c4f97d88e976da36ede. Latest consensus state: current_term: 1 leader_uuid: "228559f4c0ad4c4f97d88e976da36ede" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "228559f4c0ad4c4f97d88e976da36ede" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 45905 } } }
I20260620 23:04:42.926877 25342 sys_catalog.cc:458] T 00000000000000000000000000000000 P 228559f4c0ad4c4f97d88e976da36ede [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:42.927546 25347 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:04:42.932786 25347 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:04:42.941777 25347 catalog_manager.cc:1347] Generated new cluster ID: e7b7a2e65271450f85787545fff4c18b
I20260620 23:04:42.941921 25347 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:04:42.959132 25347 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:04:42.959575 25347 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:04:42.968854 25347 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 228559f4c0ad4c4f97d88e976da36ede: Generated new TSK 0
I20260620 23:04:42.969712 25347 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:04:42.980917 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 25263
W20260620 23:04:42.981990 25325 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:58571
W20260620 23:04:42.982729 2996 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 25263: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260620 23:04:42.982878 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 25263
2026-06-20T23:04:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1901 ms)
[----------] 18 tests from AuthTokenIssuingTest (35605 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:04:43 dist-test-slave-f63b krb5kdc[25361](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:04:43 dist-test-slave-f63b krb5kdc[25361](info): set up 2 sockets
Jun 20 23:04:43 dist-test-slave-f63b krb5kdc[25361](info): commencing operation
krb5kdc: starting...
W20260620 23:04:44.099607 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 23:04:44 dist-test-slave-f63b krb5kdc[25361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:04:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:04:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
I20260620 23:04:44.362636 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:37049
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:44699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:37049,127.2.237.61:33263,127.2.237.60:35199
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:44.694425 25374 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 23:04:44.694825 25374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:44.694955 25374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:44.695057 25374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:44.704298 25374 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:44.704422 25374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:44.704525 25374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:44.704607 25374 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:44.704708 25374 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:44.716946 25374 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:44699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:37049,127.2.237.61:33263,127.2.237.60:35199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:37049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:44.718236 25374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:44.719640 25374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:44.731559 25379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:44.731526 25382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:44.731485 25380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:44.732728 25374 server_base.cc:1034] running on GCE node
I20260620 23:04:44.733490 25374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:44.734983 25374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:44.736229 25374 hybrid_clock.cc:648] HybridClock initialized: now 1781996684736173 us; error 55 us; skew 500 ppm
I20260620 23:04:44.736557 25374 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:04:44 dist-test-slave-f63b krb5kdc[25361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996684, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:44.744444 25374 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 23:04:44.745898 25374 webserver.cc:460] Webserver started at http://127.2.237.62:45791/ using document root <none> and password file <none>
I20260620 23:04:44.746696 25374 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:44.746831 25374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:44.747185 25374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:44.750221 25374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "83c2306b89634a1c8ea385dc94c61d9b"
format_stamp: "Formatted at 2026-06-20 23:04:44 on dist-test-slave-f63b"
server_key: "7dedc2cb37cfe5f3f468f7a752b60f9b"
server_key_iv: "3f771c2b1f1fbe0512888622bf267fe8"
server_key_version: "encryptionkey@0"
I20260620 23:04:44.751068 25374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "83c2306b89634a1c8ea385dc94c61d9b"
format_stamp: "Formatted at 2026-06-20 23:04:44 on dist-test-slave-f63b"
server_key: "7dedc2cb37cfe5f3f468f7a752b60f9b"
server_key_iv: "3f771c2b1f1fbe0512888622bf267fe8"
server_key_version: "encryptionkey@0"
I20260620 23:04:44.756237 25374 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260620 23:04:44.759769 25389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:44.761188 25374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260620 23:04:44.761385 25374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "83c2306b89634a1c8ea385dc94c61d9b"
format_stamp: "Formatted at 2026-06-20 23:04:44 on dist-test-slave-f63b"
server_key: "7dedc2cb37cfe5f3f468f7a752b60f9b"
server_key_iv: "3f771c2b1f1fbe0512888622bf267fe8"
server_key_version: "encryptionkey@0"
I20260620 23:04:44.761627 25374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:44.784721 25374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:44.789343 25374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:44.789649 25374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:44.808434 25374 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:37049
I20260620 23:04:44.808459 25440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:37049 every 8 connection(s)
I20260620 23:04:44.809799 25374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:04:44.813548 25441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:44.818534 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 25374
I20260620 23:04:44.818782 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:04:44.819382 2996 external_mini_cluster.cc:1428] Setting key 57c7e8e11de5cfd9de42dd8d789c25b1
I20260620 23:04:44.820092 25441 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37049 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:04:44 dist-test-slave-f63b krb5kdc[25361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996684, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.237.61@KRBTEST.COM; defaulting to no policy
I20260620 23:04:44.844285 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:44.823305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37049 (local address 127.0.0.1:41878)
0620 23:04:44.824638 (+ 1333us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:44.824686 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:44.826404 (+ 1718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:44.827426 (+ 1022us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:44.827436 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:44.828287 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:44.828764 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:44.828789 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:44.830291 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:44.830300 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:44.832182 (+ 1882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:44.832203 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:44.833257 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:44.833269 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:44.833485 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:44.834147 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:44.834183 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:44.836785 (+ 2602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:44.840052 (+ 3267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:44.840067 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:44.840079 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:44.840445 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:44.840851 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:44.840856 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:44.840860 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:44.841022 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:44.841744 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:44.841757 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:44.841960 (+ 203us) client_negotiation.cc:769] Sending connection context
0620 23:04:44.842196 (+ 236us) client_negotiation.cc:240] Negotiation successful
0620 23:04:44.842889 (+ 693us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1074,"mutex_wait_us":190,"spinlock_wait_cycles":14592,"thread_start_us":838,"threads_started":1}
I20260620 23:04:44.844319 25443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:44.823305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41878 (local address 127.2.237.62:37049)
0620 23:04:44.824264 (+ 959us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:44.824281 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:44.824832 (+ 551us) server_negotiation.cc:407] Connection header received
0620 23:04:44.826857 (+ 2025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:44.826867 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:44.827177 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:44.827350 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:44.828977 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:44.830130 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:44.832482 (+ 2352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:44.833016 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:44.833162 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:44.837031 (+ 3869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:44.837038 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:44.837050 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:44.837081 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:44.839920 (+ 2839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:44.840576 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:44.840591 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:44.840593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:44.840732 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:44.841150 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:44.841155 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:44.841157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:44.841610 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:44.841824 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:44.842324 (+ 500us) server_negotiation.cc:299] Negotiation successful
0620 23:04:44.842780 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":678,"thread_start_us":416,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.61@KRBTEST.COM" created.
I20260620 23:04:44.847389 25441 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.61" port: 33263 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:04:44.848585 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:44.848210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:33263 (local address 127.0.0.1:51058)
0620 23:04:44.848370 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:44.848475 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.61:33263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260620 23:04:44.848953 25391 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.237.61:33263: connect: Connection refused (error 111)
I20260620 23:04:44.850445 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:44.850118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:33263 (local address 127.0.0.1:51064)
0620 23:04:44.850267 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:44.850329 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.61:33263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260620 23:04:44.850759 25441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.61:33263: Network error: Client connection negotiation failed: client connection to 127.2.237.61:33263: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.61.keytab.
Entry for principal kudu/127.2.237.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.61.keytab.
I20260620 23:04:44.877342 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.61:33263
--webserver_interface=127.2.237.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.61.keytab
--principal=kudu/127.2.237.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:44699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:37049,127.2.237.61:33263,127.2.237.60:35199
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:04:44.900104 25441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.61" port: 33263 } attempt: 1
I20260620 23:04:44.901652 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:44.901018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:33263 (local address 127.0.0.1:51076)
0620 23:04:44.901279 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:44.901434 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.61:33263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260620 23:04:44.902935 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:44.902416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:33263 (local address 127.0.0.1:51090)
0620 23:04:44.902625 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:44.902731 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.61:33263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260620 23:04:44.903162 25441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.61:33263: Network error: Client connection negotiation failed: client connection to 127.2.237.61:33263: connect: Connection refused (error 111)
I20260620 23:04:44.971407 25441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.61" port: 33263 } attempt: 2
I20260620 23:04:44.972648 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:44.972159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:33263 (local address 127.0.0.1:51106)
0620 23:04:44.972404 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:44.972506 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.61:33263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260620 23:04:44.973704 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:44.973373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:33263 (local address 127.0.0.1:51108)
0620 23:04:44.973519 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:44.973581 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.61:33263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260620 23:04:44.973951 25441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.61:33263: Network error: Client connection negotiation failed: client connection to 127.2.237.61:33263: connect: Connection refused (error 111)
I20260620 23:04:45.065327 25441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.61" port: 33263 } attempt: 3
I20260620 23:04:45.066538 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.066058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:33263 (local address 127.0.0.1:51124)
0620 23:04:45.066288 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:45.066381 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.61:33263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260620 23:04:45.067581 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.067234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:33263 (local address 127.0.0.1:51130)
0620 23:04:45.067375 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:45.067454 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.61:33263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260620 23:04:45.067759 25441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.61:33263: Network error: Client connection negotiation failed: client connection to 127.2.237.61:33263: connect: Connection refused (error 111)
I20260620 23:04:45.210989 25441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.61" port: 33263 } attempt: 4
I20260620 23:04:45.212205 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.211693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:33263 (local address 127.0.0.1:51140)
0620 23:04:45.211983 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:45.212075 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.61:33263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
W20260620 23:04:45.212984 25448 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
I20260620 23:04:45.213184 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.212843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:33263 (local address 127.0.0.1:51146)
0620 23:04:45.212990 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:45.213072 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.61:33263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260620 23:04:45.213358 25448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:45.213380 25441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.61:33263: Network error: Client connection negotiation failed: client connection to 127.2.237.61:33263: connect: Connection refused (error 111)
W20260620 23:04:45.213498 25448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:45.213637 25448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:45.221228 25448 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:45.221344 25448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:45.221473 25448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:45.221560 25448 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:45.221678 25448 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:45.231858 25448 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:44699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:37049,127.2.237.61:33263,127.2.237.60:35199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.61.keytab
--principal=kudu/127.2.237.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.61:33263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.237.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:45.233220 25448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:45.234526 25448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:45.246649 25453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:45.246637 25456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:45.246721 25454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:45.247349 25448 server_base.cc:1034] running on GCE node
I20260620 23:04:45.248190 25448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:45.249634 25448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:45.250906 25448 hybrid_clock.cc:648] HybridClock initialized: now 1781996685250839 us; error 43 us; skew 500 ppm
I20260620 23:04:45.251266 25448 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:04:45 dist-test-slave-f63b krb5kdc[25361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996685, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:45.254457 25448 init.cc:377] Logged in from keytab as kudu/127.2.237.61@KRBTEST.COM (short username kudu)
I20260620 23:04:45.255999 25448 webserver.cc:460] Webserver started at http://127.2.237.61:44681/ using document root <none> and password file <none>
I20260620 23:04:45.256707 25448 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:45.256843 25448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:45.257165 25448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:45.260206 25448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-1/data/instance:
uuid: "c17082ec4379433da24ed074db431aad"
format_stamp: "Formatted at 2026-06-20 23:04:45 on dist-test-slave-f63b"
server_key: "13882dfb088e0b82bf9de14e826573e0"
server_key_iv: "244ce49f025462a41cceb4205f23ecdc"
server_key_version: "encryptionkey@0"
I20260620 23:04:45.261027 25448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-1/wal/instance:
uuid: "c17082ec4379433da24ed074db431aad"
format_stamp: "Formatted at 2026-06-20 23:04:45 on dist-test-slave-f63b"
server_key: "13882dfb088e0b82bf9de14e826573e0"
server_key_iv: "244ce49f025462a41cceb4205f23ecdc"
server_key_version: "encryptionkey@0"
I20260620 23:04:45.266500 25448 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260620 23:04:45.270037 25463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:45.271404 25448 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260620 23:04:45.271611 25448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-1/wal
uuid: "c17082ec4379433da24ed074db431aad"
format_stamp: "Formatted at 2026-06-20 23:04:45 on dist-test-slave-f63b"
server_key: "13882dfb088e0b82bf9de14e826573e0"
server_key_iv: "244ce49f025462a41cceb4205f23ecdc"
server_key_version: "encryptionkey@0"
I20260620 23:04:45.271835 25448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:45.297550 25448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:45.302170 25448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:45.302488 25448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:45.321125 25448 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.61:33263
I20260620 23:04:45.321132 25514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.61:33263 every 8 connection(s)
I20260620 23:04:45.322602 25448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-1/data/info.pb
I20260620 23:04:45.326339 25515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:45.332753 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 25448
I20260620 23:04:45.332983 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-1/wal/instance
I20260620 23:04:45.333437 2996 external_mini_cluster.cc:1428] Setting key 39a207d122a421a895b7cb64a84f59ca
I20260620 23:04:45.332943 25515 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37049 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:04:45 dist-test-slave-f63b krb5kdc[25361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996685, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.61@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:04:45.353569 25443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.335866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41886 (local address 127.2.237.62:37049)
0620 23:04:45.336050 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:45.336055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:45.337408 (+ 1353us) server_negotiation.cc:407] Connection header received
0620 23:04:45.339434 (+ 2026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:45.339439 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:45.339556 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:45.339702 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:45.341249 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.341933 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.343529 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.343981 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.344129 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:45.347692 (+ 3563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:45.347696 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:45.347699 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:45.347733 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:45.350386 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:45.351180 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:45.351189 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:45.351192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:45.351312 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:45.351759 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:45.351764 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:45.351768 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:45.352060 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:45.352225 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:45.352989 (+ 764us) server_negotiation.cc:299] Negotiation successful
0620 23:04:45.353231 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260620 23:04:45.354681 25517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.336451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37049 (local address 127.0.0.1:41886)
0620 23:04:45.337240 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:45.337277 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:45.339147 (+ 1870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:45.339950 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:45.339958 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:45.340492 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:45.341059 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:45.341076 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.342103 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.342114 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:45.343372 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:45.343385 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.344224 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.344231 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:45.344474 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:45.345204 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:45.345226 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:45.347443 (+ 2217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:45.350538 (+ 3095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:45.350547 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:45.350564 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:45.351036 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:45.351427 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:45.351432 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:45.351435 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:45.351638 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:45.352194 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:45.352202 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:45.352601 (+ 399us) client_negotiation.cc:769] Sending connection context
0620 23:04:45.352875 (+ 274us) client_negotiation.cc:240] Negotiation successful
0620 23:04:45.353441 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":340,"threads_started":1}
I20260620 23:04:45.356467 25515 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.61" port: 33263 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.2.237.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.60@KRBTEST.COM" created.
Jun 20 23:04:45 dist-test-slave-f63b krb5kdc[25361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996685, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.61@KRBTEST.COM for kudu/127.2.237.61@KRBTEST.COM
I20260620 23:04:45.371660 25517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.357350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:33263 (local address 127.0.0.1:51148)
0620 23:04:45.357519 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:45.357535 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:45.357618 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:45.358806 (+ 1188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:45.358810 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:45.359272 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:45.359402 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:45.359410 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.360846 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.360853 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:45.361696 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:45.361706 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.362506 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.362511 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:45.362746 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:45.363356 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:45.363373 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:45.365464 (+ 2091us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:45.369208 (+ 3744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:45.369211 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:45.369214 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:45.369552 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:45.369980 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:45.369987 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:45.369992 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:45.370176 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:45.370953 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:45.370957 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:45.371079 (+ 122us) client_negotiation.cc:769] Sending connection context
0620 23:04:45.371196 (+ 117us) client_negotiation.cc:240] Negotiation successful
0620 23:04:45.371389 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":5}
I20260620 23:04:45.372159 25519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.357638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51148 (local address 127.2.237.61:33263)
0620 23:04:45.358339 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:45.358345 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:45.358371 (+ 26us) server_negotiation.cc:407] Connection header received
0620 23:04:45.358414 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:45.358419 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:45.358666 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:45.358825 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:45.359623 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.360717 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.361839 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.362367 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.362498 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:45.365688 (+ 3190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:45.365698 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:45.365711 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:45.365739 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:45.369064 (+ 3325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:45.369680 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:45.369692 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:45.369698 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:45.369841 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:45.370281 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:45.370289 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:45.370294 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:45.370821 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:45.371038 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:45.371378 (+ 340us) server_negotiation.cc:299] Negotiation successful
0620 23:04:45.371668 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":74,"server-negotiator.queue_time_us":535,"thread_start_us":338,"threads_started":1}
I20260620 23:04:45.374681 25515 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:04:45.375723 25517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.375273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:35199 (local address 127.0.0.1:44494)
0620 23:04:45.375430 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:45.375570 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:35199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260620 23:04:45.375959 25466 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.237.60:35199: connect: Connection refused (error 111)
I20260620 23:04:45.377480 25517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.377166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:35199 (local address 127.0.0.1:44496)
0620 23:04:45.377301 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:45.377382 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:35199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260620 23:04:45.377686 25515 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.60:35199: Network error: Client connection negotiation failed: client connection to 127.2.237.60:35199: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.60.keytab.
Entry for principal kudu/127.2.237.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.60.keytab.
I20260620 23:04:45.393668 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.60:35199
--webserver_interface=127.2.237.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.60.keytab
--principal=kudu/127.2.237.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:44699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:37049,127.2.237.61:33263,127.2.237.60:35199
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:04:45.426985 25515 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 } attempt: 1
I20260620 23:04:45.428254 25517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.427712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:35199 (local address 127.0.0.1:44512)
0620 23:04:45.427944 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:45.428069 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:35199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260620 23:04:45.429289 25517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.428917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:35199 (local address 127.0.0.1:44516)
0620 23:04:45.429078 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:45.429163 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:35199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260620 23:04:45.429543 25515 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.60:35199: Network error: Client connection negotiation failed: client connection to 127.2.237.60:35199: connect: Connection refused (error 111)
I20260620 23:04:45.497822 25515 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 } attempt: 2
I20260620 23:04:45.499100 25517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.498657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:35199 (local address 127.0.0.1:44528)
0620 23:04:45.498852 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:45.498946 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:35199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260620 23:04:45.500144 25517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.499828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:35199 (local address 127.0.0.1:44544)
0620 23:04:45.499956 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:45.500024 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:35199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260620 23:04:45.500452 25515 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.60:35199: Network error: Client connection negotiation failed: client connection to 127.2.237.60:35199: connect: Connection refused (error 111)
I20260620 23:04:45.512778 25441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.61" port: 33263 } attempt: 5
Jun 20 23:04:45 dist-test-slave-f63b krb5kdc[25361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996684, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for kudu/127.2.237.61@KRBTEST.COM
I20260620 23:04:45.527709 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.513521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:33263 (local address 127.0.0.1:51158)
0620 23:04:45.513740 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:45.513762 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:45.513900 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:45.514363 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:45.514367 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:45.514827 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:45.514996 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:45.515005 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.516616 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.516620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:45.517510 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:45.517520 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.518459 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.518464 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:45.518668 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:45.519296 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:45.519314 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:45.521548 (+ 2234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:45.525317 (+ 3769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:45.525322 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:45.525325 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:45.525678 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:45.526097 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:45.526100 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:45.526102 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:45.526222 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:45.526948 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:45.526953 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:45.527112 (+ 159us) client_negotiation.cc:769] Sending connection context
0620 23:04:45.527227 (+ 115us) client_negotiation.cc:240] Negotiation successful
0620 23:04:45.527440 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260620 23:04:45.527904 25519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.513744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51158 (local address 127.2.237.61:33263)
0620 23:04:45.513969 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:45.513977 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:45.514005 (+ 28us) server_negotiation.cc:407] Connection header received
0620 23:04:45.514095 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:45.514103 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:45.514200 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:45.514323 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:45.515158 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.516391 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.517656 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.518328 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.518453 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:45.521747 (+ 3294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:45.521753 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:45.521758 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:45.521798 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:45.525181 (+ 3383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:45.525800 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:45.525807 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:45.525814 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:45.525973 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:45.526332 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:45.526339 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:45.526343 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:45.526811 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:45.526990 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:45.527366 (+ 376us) server_negotiation.cc:299] Negotiation successful
0620 23:04:45.527591 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260620 23:04:45.528967 25441 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:04:45.529924 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.529580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:35199 (local address 127.0.0.1:44548)
0620 23:04:45.529727 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:45.529795 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:35199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260620 23:04:45.530784 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.530461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:35199 (local address 127.0.0.1:44558)
0620 23:04:45.530626 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:45.530698 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:35199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260620 23:04:45.531005 25441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.60:35199: Network error: Client connection negotiation failed: client connection to 127.2.237.60:35199: connect: Connection refused (error 111)
I20260620 23:04:45.582230 25441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 } attempt: 1
I20260620 23:04:45.583423 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.582962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:35199 (local address 127.0.0.1:44570)
0620 23:04:45.583160 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:45.583252 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:35199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260620 23:04:45.584435 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.584092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:35199 (local address 127.0.0.1:44580)
0620 23:04:45.584261 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:45.584332 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:35199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260620 23:04:45.584652 25441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.60:35199: Network error: Client connection negotiation failed: client connection to 127.2.237.60:35199: connect: Connection refused (error 111)
I20260620 23:04:45.591668 25515 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 } attempt: 3
I20260620 23:04:45.592710 25517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.592299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:35199 (local address 127.0.0.1:44590)
0620 23:04:45.592492 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:45.592571 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:35199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260620 23:04:45.593719 25517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.593319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:35199 (local address 127.0.0.1:44594)
0620 23:04:45.593541 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:45.593611 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:35199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
W20260620 23:04:45.593909 25515 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.60:35199: Network error: Client connection negotiation failed: client connection to 127.2.237.60:35199: connect: Connection refused (error 111)
I20260620 23:04:45.652937 25441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 } attempt: 2
I20260620 23:04:45.654119 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.653628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:35199 (local address 127.0.0.1:44602)
0620 23:04:45.653869 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:45.653955 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:35199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260620 23:04:45.655102 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.654766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:35199 (local address 127.0.0.1:44608)
0620 23:04:45.654937 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:45.655006 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:35199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260620 23:04:45.655311 25441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.60:35199: Network error: Client connection negotiation failed: client connection to 127.2.237.60:35199: connect: Connection refused (error 111)
W20260620 23:04:45.731063 25522 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 23:04:45.731451 25522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:45.731580 25522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:45.731714 25522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260620 23:04:45.737105 25515 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 } attempt: 4
I20260620 23:04:45.738265 25517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.737782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:35199 (local address 127.0.0.1:44612)
0620 23:04:45.738020 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:45.738102 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:35199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260620 23:04:45.739204 25517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.738892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:35199 (local address 127.0.0.1:44620)
0620 23:04:45.739033 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:45.739101 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:35199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260620 23:04:45.739287 25522 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:45.739418 25522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:45.739409 25515 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.60:35199: Network error: Client connection negotiation failed: client connection to 127.2.237.60:35199: connect: Connection refused (error 111)
W20260620 23:04:45.739494 25522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:45.739609 25522 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:45.739719 25522 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:45.749698 25522 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:44699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:37049,127.2.237.61:33263,127.2.237.60:35199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.60.keytab
--principal=kudu/127.2.237.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.60:35199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.237.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:45.751164 25522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:45.752452 25522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260620 23:04:45.761502 25441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 } attempt: 3
I20260620 23:04:45.762959 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.762218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:35199 (local address 127.0.0.1:44630)
0620 23:04:45.762626 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:45.762730 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:35199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223}
W20260620 23:04:45.763506 25528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:45.763844 25527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:45.764226 25530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:45.764303 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.763735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:35199 (local address 127.0.0.1:44634)
0620 23:04:45.764065 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:45.764158 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:35199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168}
W20260620 23:04:45.764561 25441 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.60:35199: Network error: Client connection negotiation failed: client connection to 127.2.237.60:35199: connect: Connection refused (error 111)
I20260620 23:04:45.764735 25522 server_base.cc:1034] running on GCE node
I20260620 23:04:45.765594 25522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:45.767072 25522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:45.768285 25522 hybrid_clock.cc:648] HybridClock initialized: now 1781996685768226 us; error 29 us; skew 500 ppm
I20260620 23:04:45.768612 25522 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:04:45 dist-test-slave-f63b krb5kdc[25361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996685, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:45.771816 25522 init.cc:377] Logged in from keytab as kudu/127.2.237.60@KRBTEST.COM (short username kudu)
I20260620 23:04:45.773296 25522 webserver.cc:460] Webserver started at http://127.2.237.60:44197/ using document root <none> and password file <none>
I20260620 23:04:45.773964 25522 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:45.774101 25522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:45.774411 25522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:45.777449 25522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-2/data/instance:
uuid: "3d93166ae77144529843acfc25aedeca"
format_stamp: "Formatted at 2026-06-20 23:04:45 on dist-test-slave-f63b"
server_key: "b37237f63b72654b074255d831641612"
server_key_iv: "7a085ab13ec0801e3dd367cbfac2e568"
server_key_version: "encryptionkey@0"
I20260620 23:04:45.778218 25522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-2/wal/instance:
uuid: "3d93166ae77144529843acfc25aedeca"
format_stamp: "Formatted at 2026-06-20 23:04:45 on dist-test-slave-f63b"
server_key: "b37237f63b72654b074255d831641612"
server_key_iv: "7a085ab13ec0801e3dd367cbfac2e568"
server_key_version: "encryptionkey@0"
I20260620 23:04:45.783298 25522 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260620 23:04:45.786805 25537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:45.788160 25522 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260620 23:04:45.788347 25522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-2/wal
uuid: "3d93166ae77144529843acfc25aedeca"
format_stamp: "Formatted at 2026-06-20 23:04:45 on dist-test-slave-f63b"
server_key: "b37237f63b72654b074255d831641612"
server_key_iv: "7a085ab13ec0801e3dd367cbfac2e568"
server_key_version: "encryptionkey@0"
I20260620 23:04:45.788580 25522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:45.830513 25522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:45.834717 25522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:45.835004 25522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:45.853752 25522 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.60:35199
I20260620 23:04:45.853775 25588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.60:35199 every 8 connection(s)
I20260620 23:04:45.855237 25522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-2/data/info.pb
I20260620 23:04:45.859045 25589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:45.859094 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 25522
I20260620 23:04:45.859268 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781996318807807-2996-0/minicluster-data/master-2/wal/instance
I20260620 23:04:45.859689 2996 external_mini_cluster.cc:1428] Setting key 99581ddc11584f612d687ff21b4e3c38
I20260620 23:04:45.874218 25589 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37049 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:04:45 dist-test-slave-f63b krb5kdc[25361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.60@KRBTEST.COM
I20260620 23:04:45.885404 25592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.861692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44638 (local address 127.2.237.60:35199)
0620 23:04:45.862631 (+ 939us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:45.862646 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:45.862699 (+ 53us) server_negotiation.cc:407] Connection header received
0620 23:04:45.864861 (+ 2162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:45.864892 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:45.865401 (+ 509us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:45.867595 (+ 2194us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:45.868125 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.869921 (+ 1796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.871614 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.872665 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.872838 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:45.877351 (+ 4513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:45.877365 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:45.877387 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:45.877432 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:45.881218 (+ 3786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:45.881855 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:45.881861 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:45.881871 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:45.882043 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:45.882436 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:45.882440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:45.882441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:45.883034 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:45.883279 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:45.883772 (+ 493us) server_negotiation.cc:299] Negotiation successful
0620 23:04:45.884186 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":462,"threads_started":1}
Jun 20 23:04:45 dist-test-slave-f63b krb5kdc[25361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996685, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.60@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:04:45.894289 25594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.877095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:37049 (local address 127.0.0.1:41892)
0620 23:04:45.878321 (+ 1226us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:45.878362 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:45.878552 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:45.879003 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:45.879008 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:45.879606 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:45.879884 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:45.879893 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.881882 (+ 1989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.881887 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:45.883356 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:45.883369 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.884304 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.884308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:45.884428 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:45.885010 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:45.885026 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:45.887316 (+ 2290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:45.891342 (+ 4026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:45.891354 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:45.891362 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:45.891827 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:45.892227 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:45.892233 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:45.892237 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:45.892369 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:45.892974 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:45.892982 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:45.893168 (+ 186us) client_negotiation.cc:769] Sending connection context
0620 23:04:45.893502 (+ 334us) client_negotiation.cc:240] Negotiation successful
0620 23:04:45.893831 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":978,"thread_start_us":796,"threads_started":1}
I20260620 23:04:45.894340 25593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.877020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41892 (local address 127.2.237.62:37049)
0620 23:04:45.877678 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:45.877687 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:45.878489 (+ 802us) server_negotiation.cc:407] Connection header received
0620 23:04:45.878715 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:45.878722 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:45.878808 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:45.878922 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:45.880040 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.881222 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.883510 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.884126 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.884256 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:45.887509 (+ 3253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:45.887514 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:45.887517 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:45.887548 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:45.891191 (+ 3643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:45.891947 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:45.891953 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:45.891956 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:45.892101 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:45.892486 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:45.892491 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:45.892494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:45.892864 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:45.893008 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:45.893646 (+ 638us) server_negotiation.cc:299] Negotiation successful
0620 23:04:45.893866 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":330,"threads_started":1}
I20260620 23:04:45.896126 25589 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.61" port: 33263 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:04:45 dist-test-slave-f63b krb5kdc[25361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
Jun 20 23:04:45 dist-test-slave-f63b krb5kdc[25361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996685, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.60@KRBTEST.COM for kudu/127.2.237.61@KRBTEST.COM
I20260620 23:04:45.904021 25595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.888530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41900 (local address 127.2.237.62:37049)
0620 23:04:45.889222 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:45.889226 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:45.889245 (+ 19us) server_negotiation.cc:407] Connection header received
0620 23:04:45.889285 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:45.889288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:45.889384 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:45.889508 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:45.891123 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.892133 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.893494 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.893957 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.894090 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:45.898638 (+ 4548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:45.898646 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:45.898655 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:45.898685 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:45.901406 (+ 2721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:45.902052 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:45.902059 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:45.902062 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:45.902187 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:45.902556 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:45.902562 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:45.902565 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:45.902846 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:45.903140 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:45.903433 (+ 293us) server_negotiation.cc:299] Negotiation successful
0620 23:04:45.903644 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":339,"threads_started":1}
I20260620 23:04:45.909832 25594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.896923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:33263 (local address 127.0.0.1:51162)
0620 23:04:45.897268 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:45.897282 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:45.897400 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:45.897711 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:45.897715 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:45.898095 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:45.898228 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:45.898235 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.899250 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.899257 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:45.900122 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:45.900132 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.900821 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.900827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:45.901027 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:45.901496 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:45.901511 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:45.903450 (+ 1939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:45.907321 (+ 3871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:45.907328 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:45.907333 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:45.907892 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:45.908273 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:45.908278 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:45.908282 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:45.908471 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:45.909082 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:45.909086 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:45.909234 (+ 148us) client_negotiation.cc:769] Sending connection context
0620 23:04:45.909352 (+ 118us) client_negotiation.cc:240] Negotiation successful
0620 23:04:45.909545 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247}
I20260620 23:04:45.910068 25519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.896926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51162 (local address 127.2.237.61:33263)
0620 23:04:45.897139 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:45.897146 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:45.897354 (+ 208us) server_negotiation.cc:407] Connection header received
0620 23:04:45.897527 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:45.897532 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:45.897592 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:45.897694 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:45.898364 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.899120 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.900269 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.900675 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.900807 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:45.903643 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:45.903647 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:45.903651 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:45.903680 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:45.907185 (+ 3505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:45.908024 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:45.908027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:45.908030 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:45.908157 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:45.908573 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:45.908576 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:45.908579 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:45.908940 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:45.909083 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:45.909483 (+ 400us) server_negotiation.cc:299] Negotiation successful
0620 23:04:45.909717 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260620 23:04:45.911204 25589 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:04:45 dist-test-slave-f63b krb5kdc[25361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.61@KRBTEST.COM
Jun 20 23:04:45 dist-test-slave-f63b krb5kdc[25361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996685, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.60@KRBTEST.COM for kudu/127.2.237.60@KRBTEST.COM
I20260620 23:04:45.922750 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.906011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51176 (local address 127.2.237.61:33263)
0620 23:04:45.906739 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:45.906749 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:45.906773 (+ 24us) server_negotiation.cc:407] Connection header received
0620 23:04:45.906823 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:45.906828 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:45.906902 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:45.907029 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:45.908552 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.909655 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.911324 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.911897 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.912027 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:45.916986 (+ 4959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:45.916990 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:45.916994 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:45.917021 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:45.919913 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:45.920746 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:45.920750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:45.920753 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:45.920874 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:45.921274 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:45.921277 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:45.921279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:45.921523 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:45.921715 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:45.922052 (+ 337us) server_negotiation.cc:299] Negotiation successful
0620 23:04:45.922293 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":323,"threads_started":1}
I20260620 23:04:45.924398 25594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.911806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:35199 (local address 127.0.0.1:44642)
0620 23:04:45.911967 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:45.911987 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:45.912070 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:45.912411 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:45.912415 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:45.912868 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:45.913001 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:45.913009 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.914387 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.914390 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:45.915373 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:45.915386 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.916137 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.916144 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:45.916396 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:45.916920 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:45.916941 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:45.918792 (+ 1851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:45.921972 (+ 3180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:45.921978 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:45.921981 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:45.922445 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:45.922847 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:45.922852 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:45.922859 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:45.923020 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:45.923523 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:45.923528 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:45.923665 (+ 137us) client_negotiation.cc:769] Sending connection context
0620 23:04:45.923808 (+ 143us) client_negotiation.cc:240] Negotiation successful
0620 23:04:45.924058 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260620 23:04:45.924817 25592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.911977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44642 (local address 127.2.237.60:35199)
0620 23:04:45.912144 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:45.912150 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:45.912167 (+ 17us) server_negotiation.cc:407] Connection header received
0620 23:04:45.912212 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:45.912217 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:45.912284 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:45.912388 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:45.913149 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.914255 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.915510 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.915955 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.916053 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:45.918956 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:45.918961 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:45.918964 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:45.918990 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:45.921820 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:45.922555 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:45.922559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:45.922561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:45.922731 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:45.923133 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:45.923137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:45.923139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:45.923402 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:45.923545 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:45.923914 (+ 369us) server_negotiation.cc:299] Negotiation successful
0620 23:04:45.924277 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":88,"server-negotiator.queue_time_us":49}
I20260620 23:04:45.927798 2996 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:04:45.929816 25589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d93166ae77144529843acfc25aedeca: Bootstrap starting.
I20260620 23:04:45.933959 25589 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d93166ae77144529843acfc25aedeca: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:45.930368 25416 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "83c2306b89634a1c8ea385dc94c61d9b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41900
I20260620 23:04:45.935312 25589 log.cc:824] T 00000000000000000000000000000000 P 3d93166ae77144529843acfc25aedeca: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:45.938335 25589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d93166ae77144529843acfc25aedeca: No bootstrap required, opened a new log
I20260620 23:04:45.941870 25441 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 } attempt: 4
Jun 20 23:04:45 dist-test-slave-f63b krb5kdc[25361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996684, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for kudu/127.2.237.60@KRBTEST.COM
I20260620 23:04:45.949481 25589 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d93166ae77144529843acfc25aedeca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c2306b89634a1c8ea385dc94c61d9b" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37049 } } peers { permanent_uuid: "c17082ec4379433da24ed074db431aad" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 33263 } } peers { permanent_uuid: "3d93166ae77144529843acfc25aedeca" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 } }
I20260620 23:04:45.950162 25589 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d93166ae77144529843acfc25aedeca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d93166ae77144529843acfc25aedeca, State: Initialized, Role: FOLLOWER
I20260620 23:04:45.950806 25589 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d93166ae77144529843acfc25aedeca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c2306b89634a1c8ea385dc94c61d9b" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37049 } } peers { permanent_uuid: "c17082ec4379433da24ed074db431aad" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 33263 } } peers { permanent_uuid: "3d93166ae77144529843acfc25aedeca" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 } }
I20260620 23:04:45.952179 25598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d93166ae77144529843acfc25aedeca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c2306b89634a1c8ea385dc94c61d9b" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37049 } } peers { permanent_uuid: "c17082ec4379433da24ed074db431aad" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 33263 } } peers { permanent_uuid: "3d93166ae77144529843acfc25aedeca" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 } } }
I20260620 23:04:45.952610 25598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d93166ae77144529843acfc25aedeca [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:04:45.953591 25589 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d93166ae77144529843acfc25aedeca [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:45.956880 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.942541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:35199 (local address 127.0.0.1:44654)
0620 23:04:45.942765 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:45.942781 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:45.942879 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:45.943222 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:45.943230 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:45.943622 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:45.943781 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:45.943790 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.944869 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.944877 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:45.945814 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:45.945825 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.946653 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.946659 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:45.946861 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:45.947499 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:45.947516 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:45.949645 (+ 2129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:45.953854 (+ 4209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:45.953859 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:45.953864 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:45.954255 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:45.954693 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:45.954700 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:45.954703 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:45.954861 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:45.955677 (+ 816us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:45.955683 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:45.955888 (+ 205us) client_negotiation.cc:769] Sending connection context
0620 23:04:45.956088 (+ 200us) client_negotiation.cc:240] Negotiation successful
0620 23:04:45.956462 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260620 23:04:45.957093 25592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:45.942639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44654 (local address 127.2.237.60:35199)
0620 23:04:45.942848 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:45.942855 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:45.942877 (+ 22us) server_negotiation.cc:407] Connection header received
0620 23:04:45.943008 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:45.943014 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:45.943080 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:45.943189 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:45.943936 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.944714 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.945966 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:45.946469 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:45.946628 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:45.950003 (+ 3375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:45.950008 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:45.950012 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:45.950043 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:45.953705 (+ 3662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:45.954399 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:45.954404 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:45.954408 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:45.954550 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:45.954989 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:45.954996 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:45.955000 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:45.955499 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:45.955737 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:45.956240 (+ 503us) server_negotiation.cc:299] Negotiation successful
0620 23:04:45.956558 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260620 23:04:45.963054 25441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83c2306b89634a1c8ea385dc94c61d9b: Bootstrap starting.
W20260620 23:04:45.965445 25609 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d93166ae77144529843acfc25aedeca: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:04:45.965658 25609 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:04:45.967398 25441 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83c2306b89634a1c8ea385dc94c61d9b: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:45.968763 25441 log.cc:824] T 00000000000000000000000000000000 P 83c2306b89634a1c8ea385dc94c61d9b: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:45.972672 25441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83c2306b89634a1c8ea385dc94c61d9b: No bootstrap required, opened a new log
I20260620 23:04:45.962998 25522 master_runner.cc:186] Error getting master registration for 127.2.237.62:37049: OK, instance_id { permanent_uuid: "83c2306b89634a1c8ea385dc94c61d9b" instance_seqno: 1781996684802872 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 23:04:45.974519 25522 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 23:04:45.980362 25441 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83c2306b89634a1c8ea385dc94c61d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c2306b89634a1c8ea385dc94c61d9b" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37049 } } peers { permanent_uuid: "c17082ec4379433da24ed074db431aad" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 33263 } } peers { permanent_uuid: "3d93166ae77144529843acfc25aedeca" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 } }
I20260620 23:04:45.980813 25441 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83c2306b89634a1c8ea385dc94c61d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83c2306b89634a1c8ea385dc94c61d9b, State: Initialized, Role: FOLLOWER
I20260620 23:04:45.981353 25441 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83c2306b89634a1c8ea385dc94c61d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c2306b89634a1c8ea385dc94c61d9b" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37049 } } peers { permanent_uuid: "c17082ec4379433da24ed074db431aad" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 33263 } } peers { permanent_uuid: "3d93166ae77144529843acfc25aedeca" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 } }
I20260620 23:04:45.982470 25612 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83c2306b89634a1c8ea385dc94c61d9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c2306b89634a1c8ea385dc94c61d9b" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37049 } } peers { permanent_uuid: "c17082ec4379433da24ed074db431aad" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 33263 } } peers { permanent_uuid: "3d93166ae77144529843acfc25aedeca" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 } } }
I20260620 23:04:45.982952 25612 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83c2306b89634a1c8ea385dc94c61d9b [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:04:45.983863 25441 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83c2306b89634a1c8ea385dc94c61d9b [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:45.983999 25416 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83c2306b89634a1c8ea385dc94c61d9b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260620 23:04:45.984264 25416 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83c2306b89634a1c8ea385dc94c61d9b [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:45.986959 25416 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83c2306b89634a1c8ea385dc94c61d9b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c2306b89634a1c8ea385dc94c61d9b" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37049 } } peers { permanent_uuid: "c17082ec4379433da24ed074db431aad" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 33263 } } peers { permanent_uuid: "3d93166ae77144529843acfc25aedeca" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 } }
I20260620 23:04:45.989113 25564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "83c2306b89634a1c8ea385dc94c61d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3d93166ae77144529843acfc25aedeca"
I20260620 23:04:45.989675 25564 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d93166ae77144529843acfc25aedeca [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:45.993347 25564 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3d93166ae77144529843acfc25aedeca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83c2306b89634a1c8ea385dc94c61d9b in term 1.
I20260620 23:04:45.993737 25416 leader_election.cc:288] T 00000000000000000000000000000000 P 83c2306b89634a1c8ea385dc94c61d9b [CANDIDATE]: Term 1 election: Requested vote from peers c17082ec4379433da24ed074db431aad (127.2.237.61:33263), 3d93166ae77144529843acfc25aedeca (127.2.237.60:35199)
I20260620 23:04:45.994037 25392 leader_election.cc:302] T 00000000000000000000000000000000 P 83c2306b89634a1c8ea385dc94c61d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d93166ae77144529843acfc25aedeca, 83c2306b89634a1c8ea385dc94c61d9b; no voters:
I20260620 23:04:45.994508 25612 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83c2306b89634a1c8ea385dc94c61d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:45.998914 25612 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83c2306b89634a1c8ea385dc94c61d9b [term 1 LEADER]: Becoming Leader. State: Replica: 83c2306b89634a1c8ea385dc94c61d9b, State: Running, Role: LEADER
I20260620 23:04:45.999773 25612 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83c2306b89634a1c8ea385dc94c61d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c2306b89634a1c8ea385dc94c61d9b" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37049 } } peers { permanent_uuid: "c17082ec4379433da24ed074db431aad" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 33263 } } peers { permanent_uuid: "3d93166ae77144529843acfc25aedeca" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 } }
I20260620 23:04:46.012460 25616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83c2306b89634a1c8ea385dc94c61d9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83c2306b89634a1c8ea385dc94c61d9b. Latest consensus state: current_term: 1 leader_uuid: "83c2306b89634a1c8ea385dc94c61d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c2306b89634a1c8ea385dc94c61d9b" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37049 } } peers { permanent_uuid: "c17082ec4379433da24ed074db431aad" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 33263 } } peers { permanent_uuid: "3d93166ae77144529843acfc25aedeca" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 } } }
I20260620 23:04:46.013147 25616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83c2306b89634a1c8ea385dc94c61d9b [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:46.025707 25622 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:04:46.038636 25515 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 } attempt: 5
W20260620 23:04:46.042562 25395 tablet.cc:2369] T 00000000000000000000000000000000 P 83c2306b89634a1c8ea385dc94c61d9b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:04:46.046403 25622 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 20 23:04:46 dist-test-slave-f63b krb5kdc[25361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996685, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.61@KRBTEST.COM for kudu/127.2.237.60@KRBTEST.COM
I20260620 23:04:46.057425 25517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:46.039357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:35199 (local address 127.0.0.1:44666)
0620 23:04:46.039777 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:46.039798 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:46.039938 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:46.040848 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:46.040854 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:46.041385 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:46.041596 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:46.041624 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:46.043391 (+ 1767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:46.043396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:46.044847 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:46.044875 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:46.046081 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:46.046087 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:46.046314 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:46.046977 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:46.047001 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:46.049879 (+ 2878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:46.054278 (+ 4399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:46.054284 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:46.054289 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:46.055022 (+ 733us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:46.055522 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:46.055527 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:46.055532 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:46.055725 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:46.056398 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:46.056403 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:46.056564 (+ 161us) client_negotiation.cc:769] Sending connection context
0620 23:04:46.056731 (+ 167us) client_negotiation.cc:240] Negotiation successful
0620 23:04:46.057021 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285}
I20260620 23:04:46.058368 25592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:46.040158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44666 (local address 127.2.237.60:35199)
0620 23:04:46.040347 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:46.040355 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:46.040376 (+ 21us) server_negotiation.cc:407] Connection header received
0620 23:04:46.040441 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:46.040446 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:46.040521 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:46.040638 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:46.042136 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:46.043212 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:46.045042 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:46.045744 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:46.045868 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:46.050106 (+ 4238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:46.050114 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:46.050120 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:46.050160 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:46.054086 (+ 3926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:46.055165 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:46.055171 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:46.055175 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:46.055381 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:46.055856 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:46.055861 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:46.055865 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:46.056182 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:46.056335 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:46.057751 (+ 1416us) server_negotiation.cc:299] Negotiation successful
0620 23:04:46.058003 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260620 23:04:46.064615 25515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c17082ec4379433da24ed074db431aad: Bootstrap starting.
I20260620 23:04:46.068970 25515 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c17082ec4379433da24ed074db431aad: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:46.070487 25515 log.cc:824] T 00000000000000000000000000000000 P c17082ec4379433da24ed074db431aad: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:46.080045 25515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c17082ec4379433da24ed074db431aad: No bootstrap required, opened a new log
W20260620 23:04:46.087129 25543 tablet.cc:2369] T 00000000000000000000000000000000 P 3d93166ae77144529843acfc25aedeca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:04:46.090207 25564 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3d93166ae77144529843acfc25aedeca [term 1 FOLLOWER]: Refusing update from remote peer 83c2306b89634a1c8ea385dc94c61d9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:04:46.091521 25616 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 83c2306b89634a1c8ea385dc94c61d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d93166ae77144529843acfc25aedeca" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:04:46.096166 25595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:46.026704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41908 (local address 127.2.237.62:37049)
0620 23:04:46.026911 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:46.026918 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:46.044000 (+ 17082us) server_negotiation.cc:407] Connection header received
0620 23:04:46.044483 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:46.044490 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:46.044569 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:46.044644 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:46.047338 (+ 2694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:46.048458 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:46.071566 (+ 23108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:46.072385 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:46.072526 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:46.084234 (+ 11708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:46.084241 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:46.084246 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:46.084291 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:46.088554 (+ 4263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:46.092398 (+ 3844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:46.092405 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:46.092408 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:46.092577 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:46.093491 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:46.093496 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:46.093499 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:46.093889 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:46.094127 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:46.095587 (+ 1460us) server_negotiation.cc:299] Negotiation successful
0620 23:04:46.095834 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260620 23:04:46.096829 25515 raft_consensus.cc:348] T 00000000000000000000000000000000 P c17082ec4379433da24ed074db431aad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c2306b89634a1c8ea385dc94c61d9b" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37049 } } peers { permanent_uuid: "c17082ec4379433da24ed074db431aad" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 33263 } } peers { permanent_uuid: "3d93166ae77144529843acfc25aedeca" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 } }
I20260620 23:04:46.101366 25637 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260620 23:04:46.101657 25515 raft_consensus.cc:729] T 00000000000000000000000000000000 P c17082ec4379433da24ed074db431aad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c17082ec4379433da24ed074db431aad, State: Initialized, Role: FOLLOWER
I20260620 23:04:46.102360 25515 consensus_queue.cc:260] T 00000000000000000000000000000000 P c17082ec4379433da24ed074db431aad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c2306b89634a1c8ea385dc94c61d9b" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37049 } } peers { permanent_uuid: "c17082ec4379433da24ed074db431aad" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 33263 } } peers { permanent_uuid: "3d93166ae77144529843acfc25aedeca" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 } }
I20260620 23:04:46.113150 25639 sys_catalog.cc:455] T 00000000000000000000000000000000 P c17082ec4379433da24ed074db431aad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c2306b89634a1c8ea385dc94c61d9b" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37049 } } peers { permanent_uuid: "c17082ec4379433da24ed074db431aad" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 33263 } } peers { permanent_uuid: "3d93166ae77144529843acfc25aedeca" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 } } }
I20260620 23:04:46.115500 25639 sys_catalog.cc:458] T 00000000000000000000000000000000 P c17082ec4379433da24ed074db431aad [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:04:46.115368 25515 sys_catalog.cc:564] T 00000000000000000000000000000000 P c17082ec4379433da24ed074db431aad [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:46.115767 25489 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c17082ec4379433da24ed074db431aad [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:46.120491 25489 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c17082ec4379433da24ed074db431aad [term 1 FOLLOWER]: Refusing update from remote peer 83c2306b89634a1c8ea385dc94c61d9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:04:46.121663 25616 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 83c2306b89634a1c8ea385dc94c61d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c17082ec4379433da24ed074db431aad" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 33263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:04:46.123073 25490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "83c2306b89634a1c8ea385dc94c61d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c17082ec4379433da24ed074db431aad"
I20260620 23:04:46.105911 25374 master_runner.cc:186] Error getting master registration for 127.2.237.61:33263: OK, instance_id { permanent_uuid: "c17082ec4379433da24ed074db431aad" instance_seqno: 1781996685315713 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 23:04:46.125628 25374 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 23:04:46.127831 25490 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c17082ec4379433da24ed074db431aad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83c2306b89634a1c8ea385dc94c61d9b in term 1.
I20260620 23:04:46.127853 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:46.028796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51192 (local address 127.2.237.61:33263)
0620 23:04:46.029035 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:46.029045 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:46.050811 (+ 21766us) server_negotiation.cc:407] Connection header received
0620 23:04:46.065283 (+ 14472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:46.065289 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:46.065411 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:46.065498 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:46.075452 (+ 9954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:46.076702 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:46.087664 (+ 10962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:46.088284 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:46.088393 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:46.108618 (+ 20225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:46.108625 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:46.108630 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:46.108677 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:46.112882 (+ 4205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:46.124626 (+ 11744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:46.124632 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:46.124636 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:46.124796 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:46.125245 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:46.125251 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:46.125255 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:46.125663 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:46.125913 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:46.127211 (+ 1298us) server_negotiation.cc:299] Negotiation successful
0620 23:04:46.127474 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260620 23:04:46.131393 25631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:46.051721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44678 (local address 127.2.237.60:35199)
0620 23:04:46.066680 (+ 14959us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:46.066686 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:46.066708 (+ 22us) server_negotiation.cc:407] Connection header received
0620 23:04:46.066765 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:46.066770 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:46.066843 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:46.066948 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:46.078765 (+ 11817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:46.080153 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:46.081976 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:46.082668 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:46.082800 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:46.099198 (+ 16398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:46.099206 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:46.099211 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:46.099254 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:46.123742 (+ 24488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:46.126391 (+ 2649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:46.126397 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:46.126401 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:46.126626 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:46.128460 (+ 1834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:46.128465 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:46.128469 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:46.128945 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:46.129251 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:46.130706 (+ 1455us) server_negotiation.cc:299] Negotiation successful
0620 23:04:46.130945 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14794,"thread_start_us":15411,"threads_started":1}
I20260620 23:04:46.139086 25639 sys_catalog.cc:455] T 00000000000000000000000000000000 P c17082ec4379433da24ed074db431aad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83c2306b89634a1c8ea385dc94c61d9b. Latest consensus state: current_term: 1 leader_uuid: "83c2306b89634a1c8ea385dc94c61d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c2306b89634a1c8ea385dc94c61d9b" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37049 } } peers { permanent_uuid: "c17082ec4379433da24ed074db431aad" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 33263 } } peers { permanent_uuid: "3d93166ae77144529843acfc25aedeca" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 } } }
I20260620 23:04:46.139405 25639 sys_catalog.cc:458] T 00000000000000000000000000000000 P c17082ec4379433da24ed074db431aad [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:04:46.142016 25633 mvcc.cc:205] Tried to move back new op lower bound from 7299058426202411008 to 7299058425867218944. Current Snapshot: MvccSnapshot[applied={T|T < 7299058426202411008}]
I20260620 23:04:46.143412 25639 sys_catalog.cc:455] T 00000000000000000000000000000000 P c17082ec4379433da24ed074db431aad [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "83c2306b89634a1c8ea385dc94c61d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c2306b89634a1c8ea385dc94c61d9b" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37049 } } peers { permanent_uuid: "c17082ec4379433da24ed074db431aad" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 33263 } } peers { permanent_uuid: "3d93166ae77144529843acfc25aedeca" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 } } }
I20260620 23:04:46.143641 25639 sys_catalog.cc:458] T 00000000000000000000000000000000 P c17082ec4379433da24ed074db431aad [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:04:46.149654 25598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d93166ae77144529843acfc25aedeca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83c2306b89634a1c8ea385dc94c61d9b. Latest consensus state: current_term: 1 leader_uuid: "83c2306b89634a1c8ea385dc94c61d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c2306b89634a1c8ea385dc94c61d9b" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37049 } } peers { permanent_uuid: "c17082ec4379433da24ed074db431aad" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 33263 } } peers { permanent_uuid: "3d93166ae77144529843acfc25aedeca" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 } } }
I20260620 23:04:46.149967 25598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d93166ae77144529843acfc25aedeca [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:04:46.150249 25622 catalog_manager.cc:1347] Generated new cluster ID: 1d4669660bb44e4bb6d5b7a83f8b7ac5
I20260620 23:04:46.150329 25622 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:04:46.165091 25616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83c2306b89634a1c8ea385dc94c61d9b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "83c2306b89634a1c8ea385dc94c61d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c2306b89634a1c8ea385dc94c61d9b" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37049 } } peers { permanent_uuid: "c17082ec4379433da24ed074db431aad" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 33263 } } peers { permanent_uuid: "3d93166ae77144529843acfc25aedeca" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 } } }
I20260620 23:04:46.165376 25616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83c2306b89634a1c8ea385dc94c61d9b [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:46.165864 25616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83c2306b89634a1c8ea385dc94c61d9b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "83c2306b89634a1c8ea385dc94c61d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c2306b89634a1c8ea385dc94c61d9b" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37049 } } peers { permanent_uuid: "c17082ec4379433da24ed074db431aad" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 33263 } } peers { permanent_uuid: "3d93166ae77144529843acfc25aedeca" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 } } }
I20260620 23:04:46.166103 25616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83c2306b89634a1c8ea385dc94c61d9b [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:46.184918 25598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d93166ae77144529843acfc25aedeca [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "83c2306b89634a1c8ea385dc94c61d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83c2306b89634a1c8ea385dc94c61d9b" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 37049 } } peers { permanent_uuid: "c17082ec4379433da24ed074db431aad" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 33263 } } peers { permanent_uuid: "3d93166ae77144529843acfc25aedeca" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 35199 } } }
I20260620 23:04:46.185217 25598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d93166ae77144529843acfc25aedeca [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:04:46.232960 25622 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:04:46.235096 25622 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:04:46.254525 25622 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83c2306b89634a1c8ea385dc94c61d9b: Generated new TSK 0
I20260620 23:04:46.255757 25622 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:04:46.283015 25657 catalog_manager.cc:1259] Loaded cluster ID: 1d4669660bb44e4bb6d5b7a83f8b7ac5
I20260620 23:04:46.283128 25657 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c17082ec4379433da24ed074db431aad: loading cluster ID for follower catalog manager: success
I20260620 23:04:46.287154 25657 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c17082ec4379433da24ed074db431aad: acquiring CA information for follower catalog manager: success
I20260620 23:04:46.289669 25657 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c17082ec4379433da24ed074db431aad: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260620 23:04:46.390098 25595 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:46.370054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41920 (local address 127.2.237.62:37049)
0620 23:04:46.370289 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:46.370296 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:46.387919 (+ 17623us) server_negotiation.cc:407] Connection header received
0620 23:04:46.388422 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:46.388429 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:46.388533 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:46.388616 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:46.389788 (+ 1172us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41920: BlockingRecv error: recv got EOF from 127.0.0.1:41920 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260620 23:04:46.392565 25596 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:46.372000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51202 (local address 127.2.237.61:33263)
0620 23:04:46.372192 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:46.372198 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:46.390919 (+ 18721us) server_negotiation.cc:407] Connection header received
0620 23:04:46.391163 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:46.391169 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:46.391274 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:46.391349 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:46.392289 (+ 940us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51202: BlockingRecv error: recv got EOF from 127.0.0.1:51202 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260620 23:04:46.396382 25631 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:46.394086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44694 (local address 127.2.237.60:35199)
0620 23:04:46.394283 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:46.394289 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:46.394723 (+ 434us) server_negotiation.cc:407] Connection header received
0620 23:04:46.394972 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:46.394977 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:46.395057 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:46.395124 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:46.396139 (+ 1015us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44694: BlockingRecv error: recv got EOF from 127.0.0.1:44694 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260620 23:04:46.443447 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:46.428098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60678 (local address 127.2.237.61:33263)
0620 23:04:46.428288 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:46.428296 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:46.435184 (+ 6888us) server_negotiation.cc:407] Connection header received
0620 23:04:46.435490 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:46.435497 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:46.435734 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:46.435896 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:46.436683 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:46.437889 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:46.440098 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:46.440917 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:46.441031 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:46.441783 (+ 752us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:46.442554 (+ 771us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:46.442707 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:46.442997 (+ 290us) server_negotiation.cc:299] Negotiation successful
0620 23:04:46.443089 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260620 23:04:46.489220 25631 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:46.478755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41908 (local address 127.2.237.60:35199)
0620 23:04:46.479002 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:46.479009 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:46.487210 (+ 8201us) server_negotiation.cc:407] Connection header received
0620 23:04:46.487490 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:46.487496 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:46.487666 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:46.487780 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:46.488936 (+ 1156us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41908: BlockingRecv error: recv got EOF from 127.0.0.1:41908 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
I20260620 23:04:46.537184 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:46.521006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60688 (local address 127.2.237.61:33263)
0620 23:04:46.521219 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:46.521227 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:46.529920 (+ 8693us) server_negotiation.cc:407] Connection header received
0620 23:04:46.530444 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:46.530453 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:46.530552 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:46.530663 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:46.531404 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:46.532577 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:46.534655 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:46.535374 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:46.535507 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:46.536189 (+ 682us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:46.536357 (+ 168us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:46.536429 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:46.536674 (+ 245us) server_negotiation.cc:299] Negotiation successful
0620 23:04:46.536768 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260620 23:04:46.581173 25631 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:46.572204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41914 (local address 127.2.237.60:35199)
0620 23:04:46.572405 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:46.572411 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:46.579585 (+ 7174us) server_negotiation.cc:407] Connection header received
0620 23:04:46.579805 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:46.579810 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:46.579918 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:46.580049 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:46.580912 (+ 863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41914: BlockingRecv error: recv got EOF from 127.0.0.1:41914 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260620 23:04:46.641671 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:46.624442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60704 (local address 127.2.237.61:33263)
0620 23:04:46.624652 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:46.624658 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:46.635295 (+ 10637us) server_negotiation.cc:407] Connection header received
0620 23:04:46.635773 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:46.635779 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:46.635883 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:46.635949 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:46.636591 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:46.637632 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:46.639483 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:46.640145 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:46.640235 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:46.640820 (+ 585us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:46.640955 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:46.641052 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:46.641289 (+ 237us) server_negotiation.cc:299] Negotiation successful
0620 23:04:46.641372 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260620 23:04:46.652734 25631 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:46.650560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41928 (local address 127.2.237.60:35199)
0620 23:04:46.650831 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:46.650841 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:46.651300 (+ 459us) server_negotiation.cc:407] Connection header received
0620 23:04:46.651505 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:46.651511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:46.651600 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:46.651713 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:46.652507 (+ 794us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41928: BlockingRecv error: recv got EOF from 127.0.0.1:41928 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260620 23:04:46.665241 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:46.659558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60714 (local address 127.2.237.61:33263)
0620 23:04:46.659857 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:46.659866 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:46.660372 (+ 506us) server_negotiation.cc:407] Connection header received
0620 23:04:46.660583 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:46.660595 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:46.660663 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:46.660758 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:46.661183 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:46.662034 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:46.663410 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:46.663883 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:46.664063 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:46.664444 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:46.664559 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:46.664666 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:46.664808 (+ 142us) server_negotiation.cc:299] Negotiation successful
0620 23:04:46.664889 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
W20260620 23:04:46.673305 25631 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:46.671149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41938 (local address 127.2.237.60:35199)
0620 23:04:46.671366 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:46.671372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:46.671950 (+ 578us) server_negotiation.cc:407] Connection header received
0620 23:04:46.672129 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:46.672135 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:46.672220 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:46.672327 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:46.673115 (+ 788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41938: BlockingRecv error: recv got EOF from 127.0.0.1:41938 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
I20260620 23:04:46.690690 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:46.685085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60720 (local address 127.2.237.61:33263)
0620 23:04:46.685318 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:46.685326 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:46.685815 (+ 489us) server_negotiation.cc:407] Connection header received
0620 23:04:46.686043 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:46.686053 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:46.686138 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:46.686245 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:46.686754 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:46.687517 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:46.688832 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:46.689315 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:46.689458 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:46.689842 (+ 384us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:46.689940 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:46.690068 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:46.690212 (+ 144us) server_negotiation.cc:299] Negotiation successful
0620 23:04:46.690289 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260620 23:04:46.700537 25631 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:46.697491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41952 (local address 127.2.237.60:35199)
0620 23:04:46.697732 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:46.697741 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:46.699139 (+ 1398us) server_negotiation.cc:407] Connection header received
0620 23:04:46.699347 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:46.699353 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:46.699439 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:46.699544 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:46.700289 (+ 745us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41952: BlockingRecv error: recv got EOF from 127.0.0.1:41952 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260620 23:04:46.727586 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:46.721921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60728 (local address 127.2.237.61:33263)
0620 23:04:46.722166 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:46.722176 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:46.722760 (+ 584us) server_negotiation.cc:407] Connection header received
0620 23:04:46.722978 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:46.722984 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:46.723067 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:46.723168 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:46.723601 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:46.724424 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:46.725797 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:46.726216 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:46.726357 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:46.726831 (+ 474us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:46.726933 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:46.727045 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:46.727163 (+ 118us) server_negotiation.cc:299] Negotiation successful
0620 23:04:46.727249 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260620 23:04:46.735755 25631 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:46.733504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41964 (local address 127.2.237.60:35199)
0620 23:04:46.733750 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:46.733758 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:46.734301 (+ 543us) server_negotiation.cc:407] Connection header received
0620 23:04:46.734510 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:46.734516 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:46.734655 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:46.734767 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:46.735523 (+ 756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41964: BlockingRecv error: recv got EOF from 127.0.0.1:41964 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260620 23:04:46.777496 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:46.771052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60734 (local address 127.2.237.61:33263)
0620 23:04:46.771302 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:46.771312 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:46.771848 (+ 536us) server_negotiation.cc:407] Connection header received
0620 23:04:46.772079 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:46.772083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:46.772160 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:46.772266 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:46.772782 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:46.773983 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:46.775444 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:46.776153 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:46.776312 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:46.776709 (+ 397us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:46.776849 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:46.776955 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:46.777107 (+ 152us) server_negotiation.cc:299] Negotiation successful
0620 23:04:46.777204 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
W20260620 23:04:46.785434 25631 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:46.783504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41966 (local address 127.2.237.60:35199)
0620 23:04:46.783716 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:46.783722 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:46.784037 (+ 315us) server_negotiation.cc:407] Connection header received
0620 23:04:46.784152 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:46.784157 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:46.784257 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:46.784375 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:46.785191 (+ 816us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41966: BlockingRecv error: recv got EOF from 127.0.0.1:41966 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260620 23:04:46.859696 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:46.853065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60738 (local address 127.2.237.61:33263)
0620 23:04:46.853343 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:46.853351 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:46.853969 (+ 618us) server_negotiation.cc:407] Connection header received
0620 23:04:46.854149 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:46.854154 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:46.854226 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:46.854317 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:46.854823 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:46.855699 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:46.857662 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:46.858187 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:46.858286 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:46.858872 (+ 586us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:46.859033 (+ 161us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:46.859109 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:46.859247 (+ 138us) server_negotiation.cc:299] Negotiation successful
0620 23:04:46.859312 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260620 23:04:46.868036 25631 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:46.865974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41972 (local address 127.2.237.60:35199)
0620 23:04:46.866177 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:46.866183 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:46.866689 (+ 506us) server_negotiation.cc:407] Connection header received
0620 23:04:46.866856 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:46.866862 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:46.866948 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:46.867047 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:46.867829 (+ 782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41972: BlockingRecv error: recv got EOF from 127.0.0.1:41972 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 23:04:46.967715 25609 catalog_manager.cc:1259] Loaded cluster ID: 1d4669660bb44e4bb6d5b7a83f8b7ac5
I20260620 23:04:46.967854 25609 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3d93166ae77144529843acfc25aedeca: loading cluster ID for follower catalog manager: success
I20260620 23:04:46.970345 25609 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3d93166ae77144529843acfc25aedeca: acquiring CA information for follower catalog manager: success
I20260620 23:04:46.972186 25609 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3d93166ae77144529843acfc25aedeca: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 23:04:47.004281 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:46.999422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60744 (local address 127.2.237.61:33263)
0620 23:04:46.999648 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:46.999653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:46.999899 (+ 246us) server_negotiation.cc:407] Connection header received
0620 23:04:47.000071 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:47.000074 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:47.000145 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:47.000221 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:47.000637 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:47.001354 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:47.002673 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:47.003117 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:47.003216 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:47.003628 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:47.003733 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:47.003834 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:47.004005 (+ 171us) server_negotiation.cc:299] Negotiation successful
0620 23:04:47.004072 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260620 23:04:47.015767 25631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:47.010076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41980 (local address 127.2.237.60:35199)
0620 23:04:47.010255 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:47.010260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:47.010751 (+ 491us) server_negotiation.cc:407] Connection header received
0620 23:04:47.010894 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:47.010898 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:47.011109 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:47.011209 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:47.011580 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:47.012331 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:47.013552 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:47.013938 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:47.014031 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:47.014487 (+ 456us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:47.015115 (+ 628us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:47.015223 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:47.015322 (+ 99us) server_negotiation.cc:299] Negotiation successful
0620 23:04:47.015385 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260620 23:04:47.029798 25631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:47.023074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41982 (local address 127.2.237.60:35199)
0620 23:04:47.023294 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:47.023303 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:47.023730 (+ 427us) server_negotiation.cc:407] Connection header received
0620 23:04:47.023960 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:47.023966 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:47.024034 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:47.024132 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:47.024755 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:47.025824 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:47.027605 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:47.028219 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:47.028337 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:47.029004 (+ 667us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:47.029147 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:47.029210 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:47.029438 (+ 228us) server_negotiation.cc:299] Negotiation successful
0620 23:04:47.029520 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 23:04:47.030030 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:47.021965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60752 (local address 127.2.237.61:33263)
0620 23:04:47.022187 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:47.022197 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:47.023660 (+ 1463us) server_negotiation.cc:407] Connection header received
0620 23:04:47.023946 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:47.023952 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:47.024062 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:47.024157 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:47.025046 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:47.025784 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:47.027889 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:47.028541 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:47.028681 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:47.029221 (+ 540us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:47.029351 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:47.029436 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:47.029648 (+ 212us) server_negotiation.cc:299] Negotiation successful
0620 23:04:47.029729 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260620 23:04:47.030890 25762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:47.021848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53766 (local address 127.2.237.62:37049)
0620 23:04:47.023926 (+ 2078us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:47.023932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:47.023956 (+ 24us) server_negotiation.cc:407] Connection header received
0620 23:04:47.024071 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:47.024076 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:47.024289 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:47.024489 (+ 200us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:47.025204 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:47.026282 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:47.028278 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:47.028743 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:47.028873 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:47.029474 (+ 601us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:47.029930 (+ 456us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:47.030037 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:47.030485 (+ 448us) server_negotiation.cc:299] Negotiation successful
0620 23:04:47.030602 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1943,"thread_start_us":1846,"threads_started":1}
I20260620 23:04:47.036216 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 25374
I20260620 23:04:47.037318 25436 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:04:47.152887 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 25374
W20260620 23:04:47.174523 25467 connection.cc:535] server connection from 127.0.0.1:51158 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 23:04:47.175194 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 25448
I20260620 23:04:47.176344 25510 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:04:47.284943 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 25448
I20260620 23:04:47.305634 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 25522
I20260620 23:04:47.306825 25584 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:04:47.414793 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 25522
2026-06-20T23:04:47Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4437 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 20 23:04:47 dist-test-slave-f63b krb5kdc[25770](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 20 23:04:47 dist-test-slave-f63b krb5kdc[25770](info): set up 2 sockets
Jun 20 23:04:47 dist-test-slave-f63b krb5kdc[25770](info): commencing operation
krb5kdc: starting...
W20260620 23:04:48.523186 2996 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.002s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/krb5kdc/test-user.keytab.
Jun 20 23:04:48 dist-test-slave-f63b krb5kdc[25770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-20T23:04:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-20T23:04:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.237.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
Entry for principal kudu/127.2.237.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab.
I20260620 23:04:48.797488 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:43431
--webserver_interface=127.2.237.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:46647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:43431,127.2.237.61:40929,127.2.237.60:36061
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260620 23:04:49.133504 25783 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 23:04:49.133914 25783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:49.134050 25783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:49.134191 25783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:49.141436 25783 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:49.141552 25783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:49.141671 25783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:49.141758 25783 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:49.141855 25783 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:49.152350 25783 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:46647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:43431,127.2.237.61:40929,127.2.237.60:36061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.62.keytab
--principal=kudu/127.2.237.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.62:43431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.237.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:49.153764 25783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:49.155206 25783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:49.167188 25788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:49.167269 25789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:49.167311 25791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:49.168592 25783 server_base.cc:1034] running on GCE node
I20260620 23:04:49.169366 25783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:49.170818 25783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:49.172071 25783 hybrid_clock.cc:648] HybridClock initialized: now 1781996689172002 us; error 34 us; skew 500 ppm
I20260620 23:04:49.172446 25783 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:04:49 dist-test-slave-f63b krb5kdc[25770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996689, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:49.175583 25783 init.cc:377] Logged in from keytab as kudu/127.2.237.62@KRBTEST.COM (short username kudu)
I20260620 23:04:49.176968 25783 webserver.cc:460] Webserver started at http://127.2.237.62:33933/ using document root <none> and password file <none>
I20260620 23:04:49.177613 25783 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:49.177695 25783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:49.177959 25783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:49.181059 25783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-0/data/instance:
uuid: "121de2598cc84b3ca1b143c0859bf96a"
format_stamp: "Formatted at 2026-06-20 23:04:49 on dist-test-slave-f63b"
server_key: "808be95ba49f8189fe0206e1903c951c"
server_key_iv: "89591ceb180593b68d246898b2c790a0"
server_key_version: "encryptionkey@0"
I20260620 23:04:49.181887 25783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-0/wal/instance:
uuid: "121de2598cc84b3ca1b143c0859bf96a"
format_stamp: "Formatted at 2026-06-20 23:04:49 on dist-test-slave-f63b"
server_key: "808be95ba49f8189fe0206e1903c951c"
server_key_iv: "89591ceb180593b68d246898b2c790a0"
server_key_version: "encryptionkey@0"
I20260620 23:04:49.187237 25783 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260620 23:04:49.190770 25798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:49.192178 25783 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260620 23:04:49.192364 25783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-0/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-0/wal
uuid: "121de2598cc84b3ca1b143c0859bf96a"
format_stamp: "Formatted at 2026-06-20 23:04:49 on dist-test-slave-f63b"
server_key: "808be95ba49f8189fe0206e1903c951c"
server_key_iv: "89591ceb180593b68d246898b2c790a0"
server_key_version: "encryptionkey@0"
I20260620 23:04:49.192596 25783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:49.206880 25783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:49.210768 25783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:49.211040 25783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:49.230873 25783 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.62:43431
I20260620 23:04:49.230892 25849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.62:43431 every 8 connection(s)
I20260620 23:04:49.232299 25783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-0/data/info.pb
I20260620 23:04:49.232700 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 25783
I20260620 23:04:49.232865 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-0/wal/instance
I20260620 23:04:49.233484 2996 external_mini_cluster.cc:1428] Setting key aaa1c3718eb5aba3d4282ccbba16bf36
I20260620 23:04:49.237234 25850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:49.247387 25850 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43431 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.2.237.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.61@KRBTEST.COM" created.
Jun 20 23:04:49 dist-test-slave-f63b krb5kdc[25770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996689, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:04:49.274755 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:49.250744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43431 (local address 127.0.0.1:43494)
0620 23:04:49.251737 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:49.251796 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:49.254695 (+ 2899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:49.255577 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:49.255586 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:49.256478 (+ 892us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:49.257001 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:49.257029 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:49.258789 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:49.258797 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:49.260781 (+ 1984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:49.260798 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:49.261789 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:49.261800 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:49.262016 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:49.262849 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:49.262872 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:49.266922 (+ 4050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:49.270315 (+ 3393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:49.270330 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:49.270346 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:49.270977 (+ 631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:49.271401 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:49.271409 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:49.271415 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:49.271647 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:49.272520 (+ 873us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:49.272532 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:49.272825 (+ 293us) client_negotiation.cc:769] Sending connection context
0620 23:04:49.273108 (+ 283us) client_negotiation.cc:240] Negotiation successful
0620 23:04:49.273751 (+ 643us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685,"mutex_wait_us":50,"thread_start_us":442,"threads_started":1}
I20260620 23:04:49.274816 25854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:49.250744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43494 (local address 127.2.237.62:43431)
0620 23:04:49.252732 (+ 1988us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:49.252738 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:49.252785 (+ 47us) server_negotiation.cc:407] Connection header received
0620 23:04:49.255088 (+ 2303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:49.255093 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:49.255333 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:49.255572 (+ 239us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:49.257230 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:49.258612 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:49.260938 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:49.261505 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:49.261639 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:49.267175 (+ 5536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:49.267184 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:49.267198 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:49.267228 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:49.270170 (+ 2942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:49.271117 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:49.271124 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:49.271127 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:49.271280 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:49.271805 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:49.271809 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:49.271811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:49.272350 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:49.272538 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:49.273248 (+ 710us) server_negotiation.cc:299] Negotiation successful
0620 23:04:49.273537 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1791,"spinlock_wait_cycles":23296,"thread_start_us":1549,"threads_started":1}
I20260620 23:04:49.278035 25850 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.61" port: 40929 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:04:49.279214 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:49.278765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:40929 (local address 127.0.0.1:47586)
0620 23:04:49.278958 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:49.279111 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.61:40929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260620 23:04:49.279495 25800 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.237.61:40929: connect: Connection refused (error 111)
I20260620 23:04:49.281065 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:49.280729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:40929 (local address 127.0.0.1:47588)
0620 23:04:49.280866 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:49.280937 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.61:40929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260620 23:04:49.281261 25850 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.61:40929: Network error: Client connection negotiation failed: client connection to 127.2.237.61:40929: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.61.keytab.
Entry for principal kudu/127.2.237.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.61.keytab.
I20260620 23:04:49.297915 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.61:40929
--webserver_interface=127.2.237.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.61.keytab
--principal=kudu/127.2.237.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:46647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:43431,127.2.237.61:40929,127.2.237.60:36061
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:04:49.330683 25850 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.61" port: 40929 } attempt: 1
I20260620 23:04:49.331851 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:49.331402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:40929 (local address 127.0.0.1:47596)
0620 23:04:49.331613 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:49.331712 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.61:40929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260620 23:04:49.332800 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:49.332465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:40929 (local address 127.0.0.1:47608)
0620 23:04:49.332614 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:49.332682 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.61:40929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260620 23:04:49.332980 25850 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.61:40929: Network error: Client connection negotiation failed: client connection to 127.2.237.61:40929: connect: Connection refused (error 111)
I20260620 23:04:49.401239 25850 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.61" port: 40929 } attempt: 2
I20260620 23:04:49.402509 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:49.402008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:40929 (local address 127.0.0.1:47616)
0620 23:04:49.402256 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:49.402349 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.61:40929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260620 23:04:49.403625 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:49.403248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:40929 (local address 127.0.0.1:47626)
0620 23:04:49.403400 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:49.403495 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.61:40929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260620 23:04:49.403811 25850 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.61:40929: Network error: Client connection negotiation failed: client connection to 127.2.237.61:40929: connect: Connection refused (error 111)
I20260620 23:04:49.495082 25850 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.61" port: 40929 } attempt: 3
I20260620 23:04:49.496268 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:49.495822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:40929 (local address 127.0.0.1:47642)
0620 23:04:49.496015 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:49.496122 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.61:40929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260620 23:04:49.497264 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:49.496904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:40929 (local address 127.0.0.1:47652)
0620 23:04:49.497069 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:49.497133 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.61:40929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260620 23:04:49.497440 25850 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.61:40929: Network error: Client connection negotiation failed: client connection to 127.2.237.61:40929: connect: Connection refused (error 111)
W20260620 23:04:49.627420 25857 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 23:04:49.627840 25857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:49.627982 25857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:49.628127 25857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260620 23:04:49.635442 25857 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:49.635576 25857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:49.635679 25857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:49.635790 25857 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:49.635885 25857 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:49.640708 25850 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.61" port: 40929 } attempt: 4
I20260620 23:04:49.642015 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:49.641444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:40929 (local address 127.0.0.1:47658)
0620 23:04:49.641780 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:49.641875 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.61:40929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192}
I20260620 23:04:49.643164 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:49.642764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:40929 (local address 127.0.0.1:47672)
0620 23:04:49.642923 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:49.643039 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.61:40929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260620 23:04:49.643363 25850 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.61:40929: Network error: Client connection negotiation failed: client connection to 127.2.237.61:40929: connect: Connection refused (error 111)
I20260620 23:04:49.646816 25857 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:46647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:43431,127.2.237.61:40929,127.2.237.60:36061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.61.keytab
--principal=kudu/127.2.237.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.61:40929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.237.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:49.648164 25857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:49.649495 25857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:49.660934 25863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:49.661077 25862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:49.661077 25865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:49.662207 25857 server_base.cc:1034] running on GCE node
I20260620 23:04:49.663054 25857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:49.664417 25857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:49.665688 25857 hybrid_clock.cc:648] HybridClock initialized: now 1781996689665617 us; error 66 us; skew 500 ppm
I20260620 23:04:49.666121 25857 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:04:49 dist-test-slave-f63b krb5kdc[25770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996689, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:49.669519 25857 init.cc:377] Logged in from keytab as kudu/127.2.237.61@KRBTEST.COM (short username kudu)
I20260620 23:04:49.671154 25857 webserver.cc:460] Webserver started at http://127.2.237.61:39705/ using document root <none> and password file <none>
I20260620 23:04:49.671916 25857 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:49.672039 25857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:49.672371 25857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:49.675451 25857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-1/data/instance:
uuid: "573fba36bb2545b7b91c3474f005e3f6"
format_stamp: "Formatted at 2026-06-20 23:04:49 on dist-test-slave-f63b"
server_key: "490c7ce22d8a54b95d88c1dc9f32be42"
server_key_iv: "22359cd863e4c44d5ce017c3877c8fb6"
server_key_version: "encryptionkey@0"
I20260620 23:04:49.676254 25857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-1/wal/instance:
uuid: "573fba36bb2545b7b91c3474f005e3f6"
format_stamp: "Formatted at 2026-06-20 23:04:49 on dist-test-slave-f63b"
server_key: "490c7ce22d8a54b95d88c1dc9f32be42"
server_key_iv: "22359cd863e4c44d5ce017c3877c8fb6"
server_key_version: "encryptionkey@0"
I20260620 23:04:49.681613 25857 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260620 23:04:49.685180 25872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:49.686679 25857 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 23:04:49.686882 25857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-1/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-1/wal
uuid: "573fba36bb2545b7b91c3474f005e3f6"
format_stamp: "Formatted at 2026-06-20 23:04:49 on dist-test-slave-f63b"
server_key: "490c7ce22d8a54b95d88c1dc9f32be42"
server_key_iv: "22359cd863e4c44d5ce017c3877c8fb6"
server_key_version: "encryptionkey@0"
I20260620 23:04:49.687129 25857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:49.708065 25857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:49.712849 25857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:49.713178 25857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:49.732097 25857 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.61:40929
I20260620 23:04:49.732122 25923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.61:40929 every 8 connection(s)
I20260620 23:04:49.733462 25857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-1/data/info.pb
I20260620 23:04:49.737475 25924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:49.743357 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 25857
I20260620 23:04:49.743551 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-1/wal/instance
I20260620 23:04:49.744102 2996 external_mini_cluster.cc:1428] Setting key 632656c807a07e9377a2ebf6b5189468
I20260620 23:04:49.744261 25924 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43431 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:04:49 dist-test-slave-f63b krb5kdc[25770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996689, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.61@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.237.60@KRBTEST.COM; defaulting to no policy
I20260620 23:04:49.770939 25854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:49.747688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43500 (local address 127.2.237.62:43431)
0620 23:04:49.747928 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:49.747936 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:49.749191 (+ 1255us) server_negotiation.cc:407] Connection header received
0620 23:04:49.751185 (+ 1994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:49.751194 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:49.751289 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:49.751416 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:49.752904 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:49.754076 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:49.755732 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:49.756521 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:49.756665 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:49.763607 (+ 6942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:49.763617 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:49.763625 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:49.763677 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:49.767308 (+ 3631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:49.768139 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:49.768149 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:49.768155 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:49.768324 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:49.768765 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:49.768771 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:49.768774 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:49.769197 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:49.769378 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:49.770217 (+ 839us) server_negotiation.cc:299] Negotiation successful
0620 23:04:49.770500 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260620 23:04:49.772009 25926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:49.748115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43431 (local address 127.0.0.1:43500)
0620 23:04:49.748995 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:49.749050 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:49.750909 (+ 1859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:49.751623 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:49.751631 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:49.752149 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:49.752716 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:49.752732 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:49.754241 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:49.754245 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:49.755563 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:49.755576 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:49.756734 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:49.756739 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:49.756950 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:49.760572 (+ 3622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:49.760597 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:49.763303 (+ 2706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:49.767489 (+ 4186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:49.767497 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:49.767511 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:49.767977 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:49.768452 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:49.768455 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:49.768457 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:49.768635 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:49.769378 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:49.769385 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:49.769756 (+ 371us) client_negotiation.cc:769] Sending connection context
0620 23:04:49.770076 (+ 320us) client_negotiation.cc:240] Negotiation successful
0620 23:04:49.770703 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":414,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.237.60@KRBTEST.COM" created.
I20260620 23:04:49.774111 25924 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.61" port: 40929 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:04:49 dist-test-slave-f63b krb5kdc[25770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996689, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.61@KRBTEST.COM for kudu/127.2.237.61@KRBTEST.COM
I20260620 23:04:49.791625 25926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:49.774963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:40929 (local address 127.0.0.1:47674)
0620 23:04:49.775272 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:49.775293 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:49.775410 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:49.776277 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:49.776289 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:49.776794 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:49.777022 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:49.777036 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:49.778437 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:49.778444 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:49.779960 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:49.779979 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:49.780857 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:49.780870 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:49.781100 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:49.781811 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:49.781840 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:49.784325 (+ 2485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:49.788815 (+ 4490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:49.788820 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:49.788822 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:49.789230 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:49.789647 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:49.789651 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:49.789653 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:49.789816 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:49.790658 (+ 842us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:49.790662 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:49.790779 (+ 117us) client_negotiation.cc:769] Sending connection context
0620 23:04:49.790918 (+ 139us) client_negotiation.cc:240] Negotiation successful
0620 23:04:49.791259 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
I20260620 23:04:49.792001 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:49.775146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47674 (local address 127.2.237.61:40929)
0620 23:04:49.775840 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:49.775846 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:49.775871 (+ 25us) server_negotiation.cc:407] Connection header received
0620 23:04:49.775916 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:49.775922 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:49.776114 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:49.776236 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:49.777186 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:49.778294 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:49.780135 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:49.780701 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:49.780801 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:49.784564 (+ 3763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:49.784582 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:49.784604 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:49.784647 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:49.788663 (+ 4016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:49.789369 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:49.789382 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:49.789388 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:49.789534 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:49.789935 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:49.789943 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:49.789949 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:49.790473 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:49.790740 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:49.791093 (+ 353us) server_negotiation.cc:299] Negotiation successful
0620 23:04:49.791421 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":537,"thread_start_us":400,"threads_started":1}
I20260620 23:04:49.794726 25924 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:04:49.795956 25926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:49.795447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:36061 (local address 127.0.0.1:48218)
0620 23:04:49.795677 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:49.795797 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:36061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260620 23:04:49.796271 25873 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.237.60:36061: connect: Connection refused (error 111)
I20260620 23:04:49.797997 25926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:49.797663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:36061 (local address 127.0.0.1:48224)
0620 23:04:49.797807 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:49.797892 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:36061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260620 23:04:49.798228 25924 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.60:36061: Network error: Client connection negotiation failed: client connection to 127.2.237.60:36061: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.237.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.60.keytab.
Entry for principal kudu/127.2.237.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.60.keytab.
I20260620 23:04:49.810246 2996 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
/tmp/dist-test-taskWPCN6h/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.60:36061
--webserver_interface=127.2.237.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.60.keytab
--principal=kudu/127.2.237.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.237.20:46647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.237.62:43431,127.2.237.61:40929,127.2.237.60:36061
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260620 23:04:49.847520 25924 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 } attempt: 1
I20260620 23:04:49.848892 25926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:49.848351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:36061 (local address 127.0.0.1:48238)
0620 23:04:49.848569 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:49.848676 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:36061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260620 23:04:49.849865 25926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:49.849518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:36061 (local address 127.0.0.1:48254)
0620 23:04:49.849660 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:49.849752 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:36061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260620 23:04:49.850068 25924 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.60:36061: Network error: Client connection negotiation failed: client connection to 127.2.237.60:36061: connect: Connection refused (error 111)
I20260620 23:04:49.918370 25924 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 } attempt: 2
I20260620 23:04:49.919579 25926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:49.919117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:36061 (local address 127.0.0.1:48266)
0620 23:04:49.919323 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:49.919417 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:36061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260620 23:04:49.920557 25926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:49.920224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:36061 (local address 127.0.0.1:48268)
0620 23:04:49.920367 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:49.920440 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:36061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260620 23:04:49.920776 25924 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.60:36061: Network error: Client connection negotiation failed: client connection to 127.2.237.60:36061: connect: Connection refused (error 111)
I20260620 23:04:49.942705 25850 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.61" port: 40929 } attempt: 5
Jun 20 23:04:49 dist-test-slave-f63b krb5kdc[25770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996689, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for kudu/127.2.237.61@KRBTEST.COM
I20260620 23:04:49.957994 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:49.943515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:40929 (local address 127.0.0.1:47690)
0620 23:04:49.943777 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:49.943799 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:49.943949 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:49.944395 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:49.944398 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:49.944784 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:49.944954 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:49.944962 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:49.946413 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:49.946417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:49.947317 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:49.947329 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:49.948432 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:49.948437 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:49.948616 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:49.949271 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:49.949289 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:49.951331 (+ 2042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:49.955469 (+ 4138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:49.955474 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:49.955477 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:49.955903 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:49.956335 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:49.956338 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:49.956341 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:49.956495 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:49.957214 (+ 719us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:49.957217 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:49.957365 (+ 148us) client_negotiation.cc:769] Sending connection context
0620 23:04:49.957490 (+ 125us) client_negotiation.cc:240] Negotiation successful
0620 23:04:49.957739 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260620 23:04:49.958223 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:49.943815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47690 (local address 127.2.237.61:40929)
0620 23:04:49.944046 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:49.944053 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:49.944076 (+ 23us) server_negotiation.cc:407] Connection header received
0620 23:04:49.944146 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:49.944151 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:49.944249 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:49.944365 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:49.945112 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:49.946263 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:49.947463 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:49.948258 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:49.948418 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:49.951534 (+ 3116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:49.951540 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:49.951544 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:49.951585 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:49.955255 (+ 3670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:49.956045 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:49.956050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:49.956055 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:49.956210 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:49.956617 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:49.956626 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:49.956630 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:49.957067 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:49.957228 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:49.957637 (+ 409us) server_negotiation.cc:299] Negotiation successful
0620 23:04:49.957873 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260620 23:04:49.959507 25850 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 } has no permanent_uuid. Determining permanent_uuid...
I20260620 23:04:49.960582 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:49.960189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:36061 (local address 127.0.0.1:48274)
0620 23:04:49.960356 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:49.960439 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:36061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260620 23:04:49.961619 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:49.961279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:36061 (local address 127.0.0.1:48288)
0620 23:04:49.961421 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:49.961509 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:36061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260620 23:04:49.961856 25850 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.60:36061: Network error: Client connection negotiation failed: client connection to 127.2.237.60:36061: connect: Connection refused (error 111)
I20260620 23:04:50.011978 25924 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 } attempt: 3
I20260620 23:04:50.013085 25850 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 } attempt: 1
I20260620 23:04:50.013484 25926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:50.012793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:36061 (local address 127.0.0.1:48296)
0620 23:04:50.013112 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:50.013266 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:36061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260620 23:04:50.014288 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:50.013746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:36061 (local address 127.0.0.1:48308)
0620 23:04:50.014032 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:50.014135 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:36061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260620 23:04:50.014942 25926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:50.014278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:36061 (local address 127.0.0.1:48310)
0620 23:04:50.014667 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:50.014768 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:36061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246}
W20260620 23:04:50.015249 25924 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.60:36061: Network error: Client connection negotiation failed: client connection to 127.2.237.60:36061: connect: Connection refused (error 111)
I20260620 23:04:50.015436 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:50.015037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:36061 (local address 127.0.0.1:48316)
0620 23:04:50.015210 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:50.015292 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:36061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260620 23:04:50.015655 25850 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.60:36061: Network error: Client connection negotiation failed: client connection to 127.2.237.60:36061: connect: Connection refused (error 111)
I20260620 23:04:50.083921 25850 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 } attempt: 2
I20260620 23:04:50.085208 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:50.084657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:36061 (local address 127.0.0.1:48326)
0620 23:04:50.084933 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:50.085040 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:36061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260620 23:04:50.086273 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:50.085916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:36061 (local address 127.0.0.1:48330)
0620 23:04:50.086080 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:50.086157 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:36061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260620 23:04:50.086505 25850 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.60:36061: Network error: Client connection negotiation failed: client connection to 127.2.237.60:36061: connect: Connection refused (error 111)
W20260620 23:04:50.152729 25931 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260620 23:04:50.153143 25931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260620 23:04:50.153270 25931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260620 23:04:50.153406 25931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260620 23:04:50.158545 25924 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 } attempt: 4
I20260620 23:04:50.159790 25926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:50.159256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:36061 (local address 127.0.0.1:48334)
0620 23:04:50.159505 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:50.159611 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:36061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260620 23:04:50.160879 25926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:50.160546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:36061 (local address 127.0.0.1:48336)
0620 23:04:50.160684 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:50.160762 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:36061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260620 23:04:50.161118 25924 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.60:36061: Network error: Client connection negotiation failed: client connection to 127.2.237.60:36061: connect: Connection refused (error 111)
W20260620 23:04:50.161588 25931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260620 23:04:50.161662 25931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260620 23:04:50.161716 25931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260620 23:04:50.161767 25931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260620 23:04:50.161831 25931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260620 23:04:50.172241 25931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.237.20:46647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.237.62:43431,127.2.237.61:40929,127.2.237.60:36061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/krb5kdc/kudu_127.2.237.60.keytab
--principal=kudu/127.2.237.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.237.60:36061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.237.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 20 Jun 2026 22:06:24 UTC on ccycloud.quasar-oxjrdg.root.comops.site
build id 4457794
ASAN enabled
I20260620 23:04:50.173559 25931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260620 23:04:50.175158 25931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260620 23:04:50.187214 25936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:50.187129 25937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260620 23:04:50.187157 25939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260620 23:04:50.192734 25850 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 } attempt: 3
I20260620 23:04:50.193521 25931 server_base.cc:1034] running on GCE node
I20260620 23:04:50.194146 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:50.193470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:36061 (local address 127.0.0.1:48344)
0620 23:04:50.193816 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:50.193915 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:36061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182}
I20260620 23:04:50.194525 25931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260620 23:04:50.195317 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:50.194907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:36061 (local address 127.0.0.1:48346)
0620 23:04:50.195134 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:50.195211 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.237.60:36061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
W20260620 23:04:50.195551 25850 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.237.60:36061: Network error: Client connection negotiation failed: client connection to 127.2.237.60:36061: connect: Connection refused (error 111)
I20260620 23:04:50.196337 25931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260620 23:04:50.197584 25931 hybrid_clock.cc:648] HybridClock initialized: now 1781996690197478 us; error 71 us; skew 500 ppm
I20260620 23:04:50.197924 25931 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 20 23:04:50 dist-test-slave-f63b krb5kdc[25770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996690, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260620 23:04:50.201145 25931 init.cc:377] Logged in from keytab as kudu/127.2.237.60@KRBTEST.COM (short username kudu)
I20260620 23:04:50.202708 25931 webserver.cc:460] Webserver started at http://127.2.237.60:43953/ using document root <none> and password file <none>
I20260620 23:04:50.203416 25931 fs_manager.cc:359] Metadata directory not provided
I20260620 23:04:50.203559 25931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260620 23:04:50.203883 25931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260620 23:04:50.207093 25931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-2/data/instance:
uuid: "3b3a416e513b449eb77ec0d16de1b301"
format_stamp: "Formatted at 2026-06-20 23:04:50 on dist-test-slave-f63b"
server_key: "cd57bcabfcfbcc647dc4678cb953df62"
server_key_iv: "74e731cedb315b515711be3807311803"
server_key_version: "encryptionkey@0"
I20260620 23:04:50.207911 25931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-2/wal/instance:
uuid: "3b3a416e513b449eb77ec0d16de1b301"
format_stamp: "Formatted at 2026-06-20 23:04:50 on dist-test-slave-f63b"
server_key: "cd57bcabfcfbcc647dc4678cb953df62"
server_key_iv: "74e731cedb315b515711be3807311803"
server_key_version: "encryptionkey@0"
I20260620 23:04:50.213084 25931 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260620 23:04:50.216660 25946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260620 23:04:50.217970 25931 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260620 23:04:50.218153 25931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-2/data,/tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-2/wal
uuid: "3b3a416e513b449eb77ec0d16de1b301"
format_stamp: "Formatted at 2026-06-20 23:04:50 on dist-test-slave-f63b"
server_key: "cd57bcabfcfbcc647dc4678cb953df62"
server_key_iv: "74e731cedb315b515711be3807311803"
server_key_version: "encryptionkey@0"
I20260620 23:04:50.218370 25931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260620 23:04:50.246960 25931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260620 23:04:50.251222 25931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260620 23:04:50.251525 25931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260620 23:04:50.270556 25931 rpc_server.cc:307] RPC server started. Bound to: 127.2.237.60:36061
I20260620 23:04:50.270653 25997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.237.60:36061 every 8 connection(s)
I20260620 23:04:50.272024 25931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-2/data/info.pb
I20260620 23:04:50.275765 25998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260620 23:04:50.277035 2996 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu as pid 25931
I20260620 23:04:50.277225 2996 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskWPCN6h/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781996318807807-2996-0/minicluster-data/master-2/wal/instance
I20260620 23:04:50.277793 2996 external_mini_cluster.cc:1428] Setting key e77d9681d6d1e64e57ee4da69379f548
I20260620 23:04:50.285537 25998 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43431 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:04:50 dist-test-slave-f63b krb5kdc[25770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.60@KRBTEST.COM
Jun 20 23:04:50 dist-test-slave-f63b krb5kdc[25770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996690, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.60@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
I20260620 23:04:50.301177 26001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:50.279675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48360 (local address 127.2.237.60:36061)
0620 23:04:50.280567 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:50.280578 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:50.280618 (+ 40us) server_negotiation.cc:407] Connection header received
0620 23:04:50.282280 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:50.282303 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:50.282794 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:50.283141 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:50.285028 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.286358 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.288136 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.289355 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.289524 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:50.293720 (+ 4196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:50.293728 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:50.293738 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:50.293773 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:50.297205 (+ 3432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:50.297914 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:50.297926 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:50.297932 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:50.298075 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:50.298443 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:50.298452 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:50.298454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:50.299010 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:50.299324 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:50.299741 (+ 417us) server_negotiation.cc:299] Negotiation successful
0620 23:04:50.300148 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":435,"threads_started":1}
I20260620 23:04:50.303682 26002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:50.288435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43514 (local address 127.2.237.62:43431)
0620 23:04:50.289038 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:50.289046 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:50.289725 (+ 679us) server_negotiation.cc:407] Connection header received
0620 23:04:50.289797 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:50.289802 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:50.289890 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:50.290012 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:50.291409 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.292262 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.293668 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.294073 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.294182 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:50.297919 (+ 3737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:50.297923 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:50.297927 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:50.297960 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:50.300798 (+ 2838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:50.301655 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:50.301662 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:50.301666 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:50.301796 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:50.302178 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:50.302183 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:50.302187 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:50.302446 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:50.302623 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:50.303075 (+ 452us) server_negotiation.cc:299] Negotiation successful
0620 23:04:50.303299 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":325,"threads_started":1}
I20260620 23:04:50.304028 26003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:50.288451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.62:43431 (local address 127.0.0.1:43514)
0620 23:04:50.289362 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:50.289396 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:50.289585 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:50.290114 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:50.290122 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:50.290975 (+ 853us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:50.291208 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:50.291216 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.292416 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.292421 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:50.293481 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:50.293510 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.294262 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.294271 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:50.294413 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:50.295032 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:50.295057 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:50.297724 (+ 2667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:50.300933 (+ 3209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:50.300943 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:50.300951 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:50.301530 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:50.301908 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:50.301913 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:50.301917 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:50.302061 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:50.302612 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:50.302620 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:50.302780 (+ 160us) client_negotiation.cc:769] Sending connection context
0620 23:04:50.302963 (+ 183us) client_negotiation.cc:240] Negotiation successful
0620 23:04:50.303259 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"thread_start_us":552,"threads_started":1}
I20260620 23:04:50.305543 25998 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.61" port: 40929 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:04:50 dist-test-slave-f63b krb5kdc[25770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.62@KRBTEST.COM
Jun 20 23:04:50 dist-test-slave-f63b krb5kdc[25770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996690, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.60@KRBTEST.COM for kudu/127.2.237.61@KRBTEST.COM
I20260620 23:04:50.319579 26002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:50.304385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43528 (local address 127.2.237.62:43431)
0620 23:04:50.304553 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:50.304558 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:50.304603 (+ 45us) server_negotiation.cc:407] Connection header received
0620 23:04:50.304658 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:50.304662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:50.304731 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:50.304802 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:50.306433 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.307397 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.308745 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.309124 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.309224 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:50.313957 (+ 4733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:50.313962 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:50.313966 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:50.314006 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:50.316657 (+ 2651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:50.317494 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:50.317499 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:50.317504 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:50.317641 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:50.318003 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:50.318011 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:50.318015 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:50.318368 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:50.318676 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:50.318979 (+ 303us) server_negotiation.cc:299] Negotiation successful
0620 23:04:50.319236 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260620 23:04:50.320006 26003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:50.306315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.61:40929 (local address 127.0.0.1:47694)
0620 23:04:50.306467 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:50.306494 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:50.306640 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:50.306980 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:50.306983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:50.307297 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:50.307430 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:50.307437 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.308420 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.308426 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:50.309607 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:50.309622 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.310540 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.310549 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:50.310795 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:50.311321 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:50.311348 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:50.313948 (+ 2600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:50.317357 (+ 3409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:50.317361 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:50.317363 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:50.317892 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:50.318285 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:50.318289 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:50.318293 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:50.318477 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:50.319147 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:50.319154 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:50.319314 (+ 160us) client_negotiation.cc:769] Sending connection context
0620 23:04:50.319455 (+ 141us) client_negotiation.cc:240] Negotiation successful
0620 23:04:50.319768 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260620 23:04:50.320880 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:50.306297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47694 (local address 127.2.237.61:40929)
0620 23:04:50.306460 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:50.306466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:50.306560 (+ 94us) server_negotiation.cc:407] Connection header received
0620 23:04:50.306760 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:50.306767 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:50.306845 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:50.306954 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:50.307617 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.308281 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.309973 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.310359 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.310453 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:50.314148 (+ 3695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:50.314153 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:50.314156 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:50.314188 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:50.317224 (+ 3036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:50.318030 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:50.318034 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:50.318036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:50.318172 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:50.318650 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:50.318656 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:50.318659 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:50.319026 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:50.319168 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:50.319593 (+ 425us) server_negotiation.cc:299] Negotiation successful
0620 23:04:50.320244 (+ 651us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260620 23:04:50.322050 25998 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 } has no permanent_uuid. Determining permanent_uuid...
Jun 20 23:04:50 dist-test-slave-f63b krb5kdc[25770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996690, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.60@KRBTEST.COM for kudu/127.2.237.60@KRBTEST.COM
Jun 20 23:04:50 dist-test-slave-f63b krb5kdc[25770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.237.61@KRBTEST.COM
I20260620 23:04:50.338100 26003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:50.322886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:36061 (local address 127.0.0.1:48372)
0620 23:04:50.323137 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:50.323150 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:50.323232 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:50.323514 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:50.323517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:50.323943 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:50.324076 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:50.324084 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.325066 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.325071 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:50.325975 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:50.325986 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.326894 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.326902 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:50.327121 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:50.327770 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:50.327802 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:50.329940 (+ 2138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:50.333123 (+ 3183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:50.333355 (+ 232us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:50.333362 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:50.334743 (+ 1381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:50.336340 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:50.336344 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:50.336348 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:50.336561 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:50.337183 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:50.337187 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:50.337356 (+ 169us) client_negotiation.cc:769] Sending connection context
0620 23:04:50.337518 (+ 162us) client_negotiation.cc:240] Negotiation successful
0620 23:04:50.337814 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177}
I20260620 23:04:50.338397 26001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:50.322791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48372 (local address 127.2.237.60:36061)
0620 23:04:50.323032 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:50.323040 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:50.323217 (+ 177us) server_negotiation.cc:407] Connection header received
0620 23:04:50.323312 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:50.323318 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:50.323396 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:50.323499 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:50.324220 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.324947 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.326119 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.326742 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.326838 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:50.330110 (+ 3272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:50.330116 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:50.330130 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:50.330166 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:50.332980 (+ 2814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:50.336032 (+ 3052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:50.336038 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:50.336044 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:50.336210 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:50.336695 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:50.336699 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:50.336703 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:50.337058 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:50.337223 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:50.337620 (+ 397us) server_negotiation.cc:299] Negotiation successful
0620 23:04:50.338010 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":119,"server-negotiator.queue_time_us":131}
I20260620 23:04:50.339341 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:50.322106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47708 (local address 127.2.237.61:40929)
0620 23:04:50.322529 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:50.322537 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:50.322556 (+ 19us) server_negotiation.cc:407] Connection header received
0620 23:04:50.322674 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:50.322679 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:50.322760 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:50.322865 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:50.324393 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.325097 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.326421 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.326976 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.327100 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:50.331536 (+ 4436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:50.331541 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:50.331544 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:50.331574 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:50.336247 (+ 4673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:50.337027 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:50.337033 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:50.337037 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:50.337198 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:50.337635 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:50.337640 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:50.337642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:50.338011 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:50.338660 (+ 649us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:50.338785 (+ 125us) server_negotiation.cc:299] Negotiation successful
0620 23:04:50.339043 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262}
I20260620 23:04:50.344725 25998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b3a416e513b449eb77ec0d16de1b301: Bootstrap starting.
I20260620 23:04:50.344887 2996 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260620 23:04:50.348620 25998 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b3a416e513b449eb77ec0d16de1b301: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:50.349704 25998 log.cc:824] T 00000000000000000000000000000000 P 3b3a416e513b449eb77ec0d16de1b301: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:50.347932 25825 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "121de2598cc84b3ca1b143c0859bf96a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43528
I20260620 23:04:50.352548 25998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b3a416e513b449eb77ec0d16de1b301: No bootstrap required, opened a new log
I20260620 23:04:50.359490 25998 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b3a416e513b449eb77ec0d16de1b301 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121de2598cc84b3ca1b143c0859bf96a" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43431 } } peers { permanent_uuid: "573fba36bb2545b7b91c3474f005e3f6" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 40929 } } peers { permanent_uuid: "3b3a416e513b449eb77ec0d16de1b301" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 } }
I20260620 23:04:50.359817 25998 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b3a416e513b449eb77ec0d16de1b301 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b3a416e513b449eb77ec0d16de1b301, State: Initialized, Role: FOLLOWER
I20260620 23:04:50.360289 25998 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b3a416e513b449eb77ec0d16de1b301 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121de2598cc84b3ca1b143c0859bf96a" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43431 } } peers { permanent_uuid: "573fba36bb2545b7b91c3474f005e3f6" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 40929 } } peers { permanent_uuid: "3b3a416e513b449eb77ec0d16de1b301" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 } }
I20260620 23:04:50.361375 26005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b3a416e513b449eb77ec0d16de1b301 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121de2598cc84b3ca1b143c0859bf96a" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43431 } } peers { permanent_uuid: "573fba36bb2545b7b91c3474f005e3f6" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 40929 } } peers { permanent_uuid: "3b3a416e513b449eb77ec0d16de1b301" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 } } }
I20260620 23:04:50.361785 26005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b3a416e513b449eb77ec0d16de1b301 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:04:50.362376 25998 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b3a416e513b449eb77ec0d16de1b301 [sys.catalog]: configured and running, proceeding with master startup.
W20260620 23:04:50.371686 26016 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3b3a416e513b449eb77ec0d16de1b301: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260620 23:04:50.371866 26016 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260620 23:04:50.372804 25850 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 } attempt: 4
I20260620 23:04:50.369650 25931 master_runner.cc:186] Error getting master registration for 127.2.237.62:43431: OK, instance_id { permanent_uuid: "121de2598cc84b3ca1b143c0859bf96a" instance_seqno: 1781996689223966 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 23:04:50.380602 25931 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Jun 20 23:04:50 dist-test-slave-f63b krb5kdc[25770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996689, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.62@KRBTEST.COM for kudu/127.2.237.60@KRBTEST.COM
I20260620 23:04:50.390403 25853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:50.373393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:36061 (local address 127.0.0.1:48376)
0620 23:04:50.373656 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:50.373676 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:50.373808 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:50.374184 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:50.374189 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:50.374745 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:50.374966 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:50.374978 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.376543 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.376549 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:50.378012 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:50.378026 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.379047 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.379054 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:50.379260 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:50.379961 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:50.379984 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:50.383931 (+ 3947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:50.387451 (+ 3520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:50.387457 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:50.387462 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:50.388023 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:50.388459 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:50.388465 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:50.388470 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:50.388655 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:50.389334 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:50.389340 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:50.389541 (+ 201us) client_negotiation.cc:769] Sending connection context
0620 23:04:50.389696 (+ 155us) client_negotiation.cc:240] Negotiation successful
0620 23:04:50.390004 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147}
I20260620 23:04:50.390407 26001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:50.373466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48376 (local address 127.2.237.60:36061)
0620 23:04:50.373683 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:50.373689 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:50.373761 (+ 72us) server_negotiation.cc:407] Connection header received
0620 23:04:50.373926 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:50.373934 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:50.374012 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:50.374129 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:50.375105 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.376371 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.378156 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.378900 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.379029 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:50.384148 (+ 5119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:50.384154 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:50.384159 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:50.384200 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:50.387311 (+ 3111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:50.388173 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:50.388179 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:50.388182 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:50.388321 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:50.388789 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:50.388794 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:50.388797 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:50.389199 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:50.389375 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:50.389814 (+ 439us) server_negotiation.cc:299] Negotiation successful
0620 23:04:50.390047 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260620 23:04:50.396458 25850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 121de2598cc84b3ca1b143c0859bf96a: Bootstrap starting.
I20260620 23:04:50.400571 25850 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 121de2598cc84b3ca1b143c0859bf96a: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:50.401937 25850 log.cc:824] T 00000000000000000000000000000000 P 121de2598cc84b3ca1b143c0859bf96a: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:50.405638 25850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 121de2598cc84b3ca1b143c0859bf96a: No bootstrap required, opened a new log
I20260620 23:04:50.413414 25850 raft_consensus.cc:348] T 00000000000000000000000000000000 P 121de2598cc84b3ca1b143c0859bf96a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121de2598cc84b3ca1b143c0859bf96a" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43431 } } peers { permanent_uuid: "573fba36bb2545b7b91c3474f005e3f6" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 40929 } } peers { permanent_uuid: "3b3a416e513b449eb77ec0d16de1b301" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 } }
I20260620 23:04:50.413857 25850 raft_consensus.cc:729] T 00000000000000000000000000000000 P 121de2598cc84b3ca1b143c0859bf96a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 121de2598cc84b3ca1b143c0859bf96a, State: Initialized, Role: FOLLOWER
I20260620 23:04:50.414358 25850 consensus_queue.cc:260] T 00000000000000000000000000000000 P 121de2598cc84b3ca1b143c0859bf96a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121de2598cc84b3ca1b143c0859bf96a" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43431 } } peers { permanent_uuid: "573fba36bb2545b7b91c3474f005e3f6" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 40929 } } peers { permanent_uuid: "3b3a416e513b449eb77ec0d16de1b301" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 } }
I20260620 23:04:50.415418 26019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 121de2598cc84b3ca1b143c0859bf96a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121de2598cc84b3ca1b143c0859bf96a" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43431 } } peers { permanent_uuid: "573fba36bb2545b7b91c3474f005e3f6" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 40929 } } peers { permanent_uuid: "3b3a416e513b449eb77ec0d16de1b301" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 } } }
I20260620 23:04:50.415827 26019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 121de2598cc84b3ca1b143c0859bf96a [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:04:50.416805 25850 sys_catalog.cc:564] T 00000000000000000000000000000000 P 121de2598cc84b3ca1b143c0859bf96a [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:50.416960 25825 raft_consensus.cc:482] T 00000000000000000000000000000000 P 121de2598cc84b3ca1b143c0859bf96a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260620 23:04:50.417171 25825 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 121de2598cc84b3ca1b143c0859bf96a [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:50.419688 25825 raft_consensus.cc:504] T 00000000000000000000000000000000 P 121de2598cc84b3ca1b143c0859bf96a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121de2598cc84b3ca1b143c0859bf96a" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43431 } } peers { permanent_uuid: "573fba36bb2545b7b91c3474f005e3f6" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 40929 } } peers { permanent_uuid: "3b3a416e513b449eb77ec0d16de1b301" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 } }
I20260620 23:04:50.421821 25973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "121de2598cc84b3ca1b143c0859bf96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3b3a416e513b449eb77ec0d16de1b301"
I20260620 23:04:50.422245 25973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b3a416e513b449eb77ec0d16de1b301 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:50.424799 25973 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3b3a416e513b449eb77ec0d16de1b301 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 121de2598cc84b3ca1b143c0859bf96a in term 1.
I20260620 23:04:50.425532 25799 leader_election.cc:302] T 00000000000000000000000000000000 P 121de2598cc84b3ca1b143c0859bf96a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 121de2598cc84b3ca1b143c0859bf96a, 3b3a416e513b449eb77ec0d16de1b301; no voters:
I20260620 23:04:50.426057 26019 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 121de2598cc84b3ca1b143c0859bf96a [term 1 FOLLOWER]: Leader election won for term 1
I20260620 23:04:50.426391 25825 leader_election.cc:288] T 00000000000000000000000000000000 P 121de2598cc84b3ca1b143c0859bf96a [CANDIDATE]: Term 1 election: Requested vote from peers 573fba36bb2545b7b91c3474f005e3f6 (127.2.237.61:40929), 3b3a416e513b449eb77ec0d16de1b301 (127.2.237.60:36061)
I20260620 23:04:50.441736 26019 raft_consensus.cc:686] T 00000000000000000000000000000000 P 121de2598cc84b3ca1b143c0859bf96a [term 1 LEADER]: Becoming Leader. State: Replica: 121de2598cc84b3ca1b143c0859bf96a, State: Running, Role: LEADER
I20260620 23:04:50.442512 26019 consensus_queue.cc:237] T 00000000000000000000000000000000 P 121de2598cc84b3ca1b143c0859bf96a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121de2598cc84b3ca1b143c0859bf96a" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43431 } } peers { permanent_uuid: "573fba36bb2545b7b91c3474f005e3f6" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 40929 } } peers { permanent_uuid: "3b3a416e513b449eb77ec0d16de1b301" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 } }
I20260620 23:04:50.451222 26024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 121de2598cc84b3ca1b143c0859bf96a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 121de2598cc84b3ca1b143c0859bf96a. Latest consensus state: current_term: 1 leader_uuid: "121de2598cc84b3ca1b143c0859bf96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121de2598cc84b3ca1b143c0859bf96a" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43431 } } peers { permanent_uuid: "573fba36bb2545b7b91c3474f005e3f6" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 40929 } } peers { permanent_uuid: "3b3a416e513b449eb77ec0d16de1b301" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 } } }
I20260620 23:04:50.451679 26024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 121de2598cc84b3ca1b143c0859bf96a [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:50.458061 26031 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260620 23:04:50.460312 25924 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 } attempt: 5
W20260620 23:04:50.463840 25804 tablet.cc:2369] T 00000000000000000000000000000000 P 121de2598cc84b3ca1b143c0859bf96a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:04:50.472766 26031 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260620 23:04:50.494508 26042 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
Jun 20 23:04:50 dist-test-slave-f63b krb5kdc[25770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781996689, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.237.61@KRBTEST.COM for kudu/127.2.237.60@KRBTEST.COM
W20260620 23:04:50.503623 25952 tablet.cc:2369] T 00000000000000000000000000000000 P 3b3a416e513b449eb77ec0d16de1b301: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260620 23:04:50.508460 25973 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3b3a416e513b449eb77ec0d16de1b301 [term 1 FOLLOWER]: Refusing update from remote peer 121de2598cc84b3ca1b143c0859bf96a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:04:50.509502 26024 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 121de2598cc84b3ca1b143c0859bf96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3a416e513b449eb77ec0d16de1b301" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:04:50.525056 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:50.453870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47710 (local address 127.2.237.61:40929)
0620 23:04:50.454067 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:50.454074 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:50.469106 (+ 15032us) server_negotiation.cc:407] Connection header received
0620 23:04:50.469764 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:50.469773 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:50.469883 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:50.469977 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:50.478757 (+ 8780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.480284 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.482738 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.483693 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.483824 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:50.509643 (+ 25819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:50.509651 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:50.509656 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:50.509700 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:50.513357 (+ 3657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:50.515945 (+ 2588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:50.515951 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:50.515955 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:50.516731 (+ 776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:50.517458 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:50.517463 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:50.517467 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:50.518052 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:50.518433 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:50.524026 (+ 5593us) server_negotiation.cc:299] Negotiation successful
0620 23:04:50.524436 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260620 23:04:50.529976 26005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b3a416e513b449eb77ec0d16de1b301 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 121de2598cc84b3ca1b143c0859bf96a. Latest consensus state: current_term: 1 leader_uuid: "121de2598cc84b3ca1b143c0859bf96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121de2598cc84b3ca1b143c0859bf96a" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43431 } } peers { permanent_uuid: "573fba36bb2545b7b91c3474f005e3f6" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 40929 } } peers { permanent_uuid: "3b3a416e513b449eb77ec0d16de1b301" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 } } }
I20260620 23:04:50.530350 26005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b3a416e513b449eb77ec0d16de1b301 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:04:50.534344 26002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:50.455000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43542 (local address 127.2.237.62:43431)
0620 23:04:50.455189 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:50.455196 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:50.474157 (+ 18961us) server_negotiation.cc:407] Connection header received
0620 23:04:50.474590 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:50.474597 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:50.474713 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:50.474814 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:50.480793 (+ 5979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.482307 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.498260 (+ 15953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.498993 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.499098 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:50.519096 (+ 19998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:50.519104 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:50.519111 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:50.519160 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:50.523134 (+ 3974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:50.529416 (+ 6282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:50.529423 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:50.529427 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:50.529603 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:50.530967 (+ 1364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:50.530973 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:50.530976 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:50.531388 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:50.531734 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:50.533624 (+ 1890us) server_negotiation.cc:299] Negotiation successful
0620 23:04:50.533879 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260620 23:04:50.534504 26043 mvcc.cc:205] Tried to move back new op lower bound from 7299058444308963328 to 7299058444060131328. Current Snapshot: MvccSnapshot[applied={T|T < 7299058444308963328}]
I20260620 23:04:50.537093 26024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 121de2598cc84b3ca1b143c0859bf96a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "121de2598cc84b3ca1b143c0859bf96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121de2598cc84b3ca1b143c0859bf96a" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43431 } } peers { permanent_uuid: "573fba36bb2545b7b91c3474f005e3f6" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 40929 } } peers { permanent_uuid: "3b3a416e513b449eb77ec0d16de1b301" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 } } }
I20260620 23:04:50.537392 26024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 121de2598cc84b3ca1b143c0859bf96a [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:50.539562 26031 catalog_manager.cc:1347] Generated new cluster ID: 1ebc5e2167254c3b923754b637f61947
I20260620 23:04:50.539667 26031 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260620 23:04:50.540091 26044 mvcc.cc:205] Tried to move back new op lower bound from 7299058444308963328 to 7299058444060131328. Current Snapshot: MvccSnapshot[applied={T|T < 7299058444308963328}]
I20260620 23:04:50.541618 26005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b3a416e513b449eb77ec0d16de1b301 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "121de2598cc84b3ca1b143c0859bf96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121de2598cc84b3ca1b143c0859bf96a" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43431 } } peers { permanent_uuid: "573fba36bb2545b7b91c3474f005e3f6" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 40929 } } peers { permanent_uuid: "3b3a416e513b449eb77ec0d16de1b301" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 } } }
I20260620 23:04:50.541903 26005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b3a416e513b449eb77ec0d16de1b301 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:04:50.499449 25783 master_runner.cc:186] Error getting master registration for 127.2.237.61:40929: OK, instance_id { permanent_uuid: "573fba36bb2545b7b91c3474f005e3f6" instance_seqno: 1781996689726691 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260620 23:04:50.560468 25783 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260620 23:04:50.559520 26031 catalog_manager.cc:1370] Generated new certificate authority record
I20260620 23:04:50.563752 26001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:50.457835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48386 (local address 127.2.237.60:36061)
0620 23:04:50.458009 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:50.458017 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:50.483618 (+ 25601us) server_negotiation.cc:407] Connection header received
0620 23:04:50.484273 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:50.484282 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:50.484376 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:50.484453 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:50.501746 (+ 17293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.503016 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.506976 (+ 3960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.507642 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.507764 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:50.548569 (+ 40805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:50.548576 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:50.548581 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:50.548627 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:50.552177 (+ 3550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:50.557523 (+ 5346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:50.557529 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:50.557532 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:50.557708 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:50.560311 (+ 2603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:50.560316 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:50.560319 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:50.560724 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:50.561014 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:50.563110 (+ 2096us) server_negotiation.cc:299] Negotiation successful
0620 23:04:50.563377 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260620 23:04:50.564415 26031 catalog_manager.cc:1504] Loading token signing keys...
I20260620 23:04:50.567746 25926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:50.460932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.237.60:36061 (local address 127.0.0.1:48394)
0620 23:04:50.461211 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0620 23:04:50.461231 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0620 23:04:50.461352 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0620 23:04:50.487905 (+ 26553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0620 23:04:50.487914 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0620 23:04:50.488438 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0620 23:04:50.488648 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:50.488662 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.492434 (+ 3772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.492442 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:50.494336 (+ 1894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0620 23:04:50.494360 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.496312 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.496320 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0620 23:04:50.496572 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:50.497276 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0620 23:04:50.497302 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0620 23:04:50.500280 (+ 2978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0620 23:04:50.562058 (+ 61778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:50.562066 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:50.562072 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:50.562728 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:50.565682 (+ 2954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0620 23:04:50.565688 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0620 23:04:50.565691 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0620 23:04:50.565895 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0620 23:04:50.566657 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0620 23:04:50.566663 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0620 23:04:50.566842 (+ 179us) client_negotiation.cc:769] Sending connection context
0620 23:04:50.566993 (+ 151us) client_negotiation.cc:240] Negotiation successful
0620 23:04:50.567318 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159}
I20260620 23:04:50.569026 26038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:50.461751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48394 (local address 127.2.237.60:36061)
0620 23:04:50.487498 (+ 25747us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:50.487504 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:50.487529 (+ 25us) server_negotiation.cc:407] Connection header received
0620 23:04:50.487591 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:50.487596 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:50.487727 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:50.487853 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:50.490888 (+ 3035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.492250 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.494601 (+ 2351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.496118 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.497027 (+ 909us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:50.500613 (+ 3586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0620 23:04:50.500622 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0620 23:04:50.500627 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0620 23:04:50.500675 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0620 23:04:50.561454 (+ 60779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:50.562894 (+ 1440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:50.565289 (+ 2395us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:50.565299 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:50.565511 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0620 23:04:50.566067 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0620 23:04:50.566076 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0620 23:04:50.566081 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0620 23:04:50.566472 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0620 23:04:50.568242 (+ 1770us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:50.568369 (+ 127us) server_negotiation.cc:299] Negotiation successful
0620 23:04:50.568605 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25584,"thread_start_us":26298,"threads_started":1}
I20260620 23:04:50.575152 25924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 573fba36bb2545b7b91c3474f005e3f6: Bootstrap starting.
I20260620 23:04:50.587617 26031 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 121de2598cc84b3ca1b143c0859bf96a: Generated new TSK 0
I20260620 23:04:50.588734 26031 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260620 23:04:50.589033 25924 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 573fba36bb2545b7b91c3474f005e3f6: Neither blocks nor log segments found. Creating new log.
I20260620 23:04:50.592490 25924 log.cc:824] T 00000000000000000000000000000000 P 573fba36bb2545b7b91c3474f005e3f6: Log is configured to *not* fsync() on all Append() calls
I20260620 23:04:50.605120 25924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 573fba36bb2545b7b91c3474f005e3f6: No bootstrap required, opened a new log
I20260620 23:04:50.618683 25924 raft_consensus.cc:348] T 00000000000000000000000000000000 P 573fba36bb2545b7b91c3474f005e3f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121de2598cc84b3ca1b143c0859bf96a" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43431 } } peers { permanent_uuid: "573fba36bb2545b7b91c3474f005e3f6" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 40929 } } peers { permanent_uuid: "3b3a416e513b449eb77ec0d16de1b301" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 } }
I20260620 23:04:50.619282 25924 raft_consensus.cc:729] T 00000000000000000000000000000000 P 573fba36bb2545b7b91c3474f005e3f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 573fba36bb2545b7b91c3474f005e3f6, State: Initialized, Role: FOLLOWER
I20260620 23:04:50.619913 25924 consensus_queue.cc:260] T 00000000000000000000000000000000 P 573fba36bb2545b7b91c3474f005e3f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121de2598cc84b3ca1b143c0859bf96a" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43431 } } peers { permanent_uuid: "573fba36bb2545b7b91c3474f005e3f6" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 40929 } } peers { permanent_uuid: "3b3a416e513b449eb77ec0d16de1b301" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 } }
I20260620 23:04:50.635241 26050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 573fba36bb2545b7b91c3474f005e3f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121de2598cc84b3ca1b143c0859bf96a" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43431 } } peers { permanent_uuid: "573fba36bb2545b7b91c3474f005e3f6" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 40929 } } peers { permanent_uuid: "3b3a416e513b449eb77ec0d16de1b301" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 } } }
I20260620 23:04:50.635867 26050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 573fba36bb2545b7b91c3474f005e3f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:04:50.636785 25924 sys_catalog.cc:564] T 00000000000000000000000000000000 P 573fba36bb2545b7b91c3474f005e3f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260620 23:04:50.637310 25898 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 573fba36bb2545b7b91c3474f005e3f6 [term 0 FOLLOWER]: Advancing to term 1
I20260620 23:04:50.642256 25898 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 573fba36bb2545b7b91c3474f005e3f6 [term 1 FOLLOWER]: Refusing update from remote peer 121de2598cc84b3ca1b143c0859bf96a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260620 23:04:50.642686 25899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "121de2598cc84b3ca1b143c0859bf96a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "573fba36bb2545b7b91c3474f005e3f6"
I20260620 23:04:50.647395 25899 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 573fba36bb2545b7b91c3474f005e3f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 121de2598cc84b3ca1b143c0859bf96a in term 1.
I20260620 23:04:50.649689 26019 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 121de2598cc84b3ca1b143c0859bf96a [LEADER]: Connected to new peer: Peer: permanent_uuid: "573fba36bb2545b7b91c3474f005e3f6" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 40929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260620 23:04:50.697069 26050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 573fba36bb2545b7b91c3474f005e3f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 121de2598cc84b3ca1b143c0859bf96a. Latest consensus state: current_term: 1 leader_uuid: "121de2598cc84b3ca1b143c0859bf96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121de2598cc84b3ca1b143c0859bf96a" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43431 } } peers { permanent_uuid: "573fba36bb2545b7b91c3474f005e3f6" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 40929 } } peers { permanent_uuid: "3b3a416e513b449eb77ec0d16de1b301" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 } } }
I20260620 23:04:50.697455 26050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 573fba36bb2545b7b91c3474f005e3f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:04:50.704979 26058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 573fba36bb2545b7b91c3474f005e3f6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "121de2598cc84b3ca1b143c0859bf96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121de2598cc84b3ca1b143c0859bf96a" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43431 } } peers { permanent_uuid: "573fba36bb2545b7b91c3474f005e3f6" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 40929 } } peers { permanent_uuid: "3b3a416e513b449eb77ec0d16de1b301" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 } } }
I20260620 23:04:50.711671 26058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 573fba36bb2545b7b91c3474f005e3f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260620 23:04:50.715283 26067 catalog_manager.cc:1259] Loaded cluster ID: 1ebc5e2167254c3b923754b637f61947
I20260620 23:04:50.715453 26067 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 573fba36bb2545b7b91c3474f005e3f6: loading cluster ID for follower catalog manager: success
I20260620 23:04:50.719417 26067 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 573fba36bb2545b7b91c3474f005e3f6: acquiring CA information for follower catalog manager: success
I20260620 23:04:50.722256 26067 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 573fba36bb2545b7b91c3474f005e3f6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260620 23:04:50.743304 2996 mini_kdc.cc:348] Time spent kdestroy: real 0.147s user 0.000s sys 0.027s
W20260620 23:04:50.752099 26002 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:50.747008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43552 (local address 127.2.237.62:43431)
0620 23:04:50.747248 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:50.747255 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:50.748930 (+ 1675us) server_negotiation.cc:407] Connection header received
0620 23:04:50.749756 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:50.749762 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:50.749859 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:50.749941 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:50.751846 (+ 1905us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43552: BlockingRecv error: recv got EOF from 127.0.0.1:43552 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260620 23:04:50.753571 25928 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:50.750185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47714 (local address 127.2.237.61:40929)
0620 23:04:50.750375 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:50.750381 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:50.750401 (+ 20us) server_negotiation.cc:407] Connection header received
0620 23:04:50.750459 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:50.750463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:50.750559 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:50.750680 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:50.753327 (+ 2647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47714: BlockingRecv error: recv got EOF from 127.0.0.1:47714 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260620 23:04:50.758169 26038 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:50.748187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48398 (local address 127.2.237.60:36061)
0620 23:04:50.748357 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:50.748363 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:50.755926 (+ 7563us) server_negotiation.cc:407] Connection header received
0620 23:04:50.756742 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:50.756748 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:50.756858 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:50.756943 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:50.757928 (+ 985us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48398: BlockingRecv error: recv got EOF from 127.0.0.1:48398 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260620 23:04:50.818815 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:50.799352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47718 (local address 127.2.237.61:40929)
0620 23:04:50.799555 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:50.799562 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:50.810164 (+ 10602us) server_negotiation.cc:407] Connection header received
0620 23:04:50.810792 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:50.810799 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:50.811003 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:50.811173 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:50.811948 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.813102 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.815236 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.815916 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.816030 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:50.816909 (+ 879us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:50.817852 (+ 943us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:50.818009 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:50.818330 (+ 321us) server_negotiation.cc:299] Negotiation successful
0620 23:04:50.818436 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260620 23:04:50.861230 26038 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:50.851951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48404 (local address 127.2.237.60:36061)
0620 23:04:50.852171 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:50.852181 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:50.859235 (+ 7054us) server_negotiation.cc:407] Connection header received
0620 23:04:50.859520 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:50.859526 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:50.859661 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:50.859772 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:50.860914 (+ 1142us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48404: BlockingRecv error: recv got EOF from 127.0.0.1:48404 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260620 23:04:50.927597 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:50.909032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47724 (local address 127.2.237.61:40929)
0620 23:04:50.909262 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:50.909268 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:50.920108 (+ 10840us) server_negotiation.cc:407] Connection header received
0620 23:04:50.920705 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:50.920712 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:50.920839 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:50.920918 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:50.921745 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.922981 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.925165 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:50.925859 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:50.925967 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:50.926672 (+ 705us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:50.926833 (+ 161us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:50.926903 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:50.927125 (+ 222us) server_negotiation.cc:299] Negotiation successful
0620 23:04:50.927229 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260620 23:04:50.976050 26038 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:50.961643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48408 (local address 127.2.237.60:36061)
0620 23:04:50.961872 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:50.961879 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:50.973907 (+ 12028us) server_negotiation.cc:407] Connection header received
0620 23:04:50.974481 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:50.974488 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:50.974613 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:50.974700 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:50.975792 (+ 1092us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48408: BlockingRecv error: recv got EOF from 127.0.0.1:48408 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260620 23:04:51.028848 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:51.014732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47726 (local address 127.2.237.61:40929)
0620 23:04:51.014929 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:51.014939 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:51.021760 (+ 6821us) server_negotiation.cc:407] Connection header received
0620 23:04:51.022000 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:51.022007 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:51.022104 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:51.022208 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:51.023078 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:51.024237 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:51.026481 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:51.027205 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:51.027314 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:51.027944 (+ 630us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:51.028120 (+ 176us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:51.028191 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:51.028405 (+ 214us) server_negotiation.cc:299] Negotiation successful
0620 23:04:51.028499 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260620 23:04:51.072039 26038 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:51.063295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48420 (local address 127.2.237.60:36061)
0620 23:04:51.063516 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:51.063523 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:51.070469 (+ 6946us) server_negotiation.cc:407] Connection header received
0620 23:04:51.070791 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:51.070798 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:51.070894 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:51.071017 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:51.071829 (+ 812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48420: BlockingRecv error: recv got EOF from 127.0.0.1:48420 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260620 23:04:51.086264 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:51.080208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47728 (local address 127.2.237.61:40929)
0620 23:04:51.080540 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:51.080549 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:51.081100 (+ 551us) server_negotiation.cc:407] Connection header received
0620 23:04:51.081330 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:51.081340 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:51.081420 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:51.081527 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:51.081958 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:51.082869 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:51.084308 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:51.084925 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:51.085065 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:51.085480 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:51.085604 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:51.085706 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:51.085859 (+ 153us) server_negotiation.cc:299] Negotiation successful
0620 23:04:51.085943 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
W20260620 23:04:51.102427 26038 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:51.100051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48424 (local address 127.2.237.60:36061)
0620 23:04:51.100367 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:51.100374 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:51.100932 (+ 558us) server_negotiation.cc:407] Connection header received
0620 23:04:51.101155 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:51.101161 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:51.101253 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:51.101373 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:51.102172 (+ 799us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48424: BlockingRecv error: recv got EOF from 127.0.0.1:48424 (error 108)
Metrics: {"server-negotiator.queue_time_us":151}
I20260620 23:04:51.119887 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:51.113897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47730 (local address 127.2.237.61:40929)
0620 23:04:51.114160 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:51.114169 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:51.114774 (+ 605us) server_negotiation.cc:407] Connection header received
0620 23:04:51.115015 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:51.115021 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:51.115103 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:51.115211 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:51.115651 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:51.116388 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:51.117863 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:51.118462 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:51.118625 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:51.119071 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:51.119191 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:51.119299 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:51.119437 (+ 138us) server_negotiation.cc:299] Negotiation successful
0620 23:04:51.119523 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260620 23:04:51.128274 26038 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:51.126057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48434 (local address 127.2.237.60:36061)
0620 23:04:51.126313 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:51.126322 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:51.126855 (+ 533us) server_negotiation.cc:407] Connection header received
0620 23:04:51.127071 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:51.127076 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:51.127168 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:51.127277 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:51.128054 (+ 777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48434: BlockingRecv error: recv got EOF from 127.0.0.1:48434 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260620 23:04:51.154107 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:51.148149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47740 (local address 127.2.237.61:40929)
0620 23:04:51.148363 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:51.148372 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:51.148696 (+ 324us) server_negotiation.cc:407] Connection header received
0620 23:04:51.148839 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:51.148845 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:51.148947 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:51.149100 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:51.149504 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:51.150766 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:51.152099 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:51.152795 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:51.152932 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:51.153366 (+ 434us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:51.153499 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:51.153597 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:51.153724 (+ 127us) server_negotiation.cc:299] Negotiation successful
0620 23:04:51.153812 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260620 23:04:51.162103 26038 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:51.160167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48438 (local address 127.2.237.60:36061)
0620 23:04:51.160379 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:51.160388 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:51.160661 (+ 273us) server_negotiation.cc:407] Connection header received
0620 23:04:51.160868 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:51.160874 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:51.160972 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:51.161081 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:51.161840 (+ 759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48438: BlockingRecv error: recv got EOF from 127.0.0.1:48438 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260620 23:04:51.183386 26019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 121de2598cc84b3ca1b143c0859bf96a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "121de2598cc84b3ca1b143c0859bf96a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "121de2598cc84b3ca1b143c0859bf96a" member_type: VOTER last_known_addr { host: "127.2.237.62" port: 43431 } } peers { permanent_uuid: "573fba36bb2545b7b91c3474f005e3f6" member_type: VOTER last_known_addr { host: "127.2.237.61" port: 40929 } } peers { permanent_uuid: "3b3a416e513b449eb77ec0d16de1b301" member_type: VOTER last_known_addr { host: "127.2.237.60" port: 36061 } } }
I20260620 23:04:51.183784 26019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 121de2598cc84b3ca1b143c0859bf96a [sys.catalog]: This master's current role is: LEADER
I20260620 23:04:51.204115 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:51.197910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47744 (local address 127.2.237.61:40929)
0620 23:04:51.198131 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:51.198138 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:51.198592 (+ 454us) server_negotiation.cc:407] Connection header received
0620 23:04:51.198841 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:51.198848 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:51.199011 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:51.199150 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:51.199568 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:51.200790 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:51.202173 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:51.202728 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:51.202882 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:51.203316 (+ 434us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:51.203432 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:51.203534 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:51.203692 (+ 158us) server_negotiation.cc:299] Negotiation successful
0620 23:04:51.203784 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260620 23:04:51.212312 26038 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:51.210288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48442 (local address 127.2.237.60:36061)
0620 23:04:51.210518 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:51.210526 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:51.210907 (+ 381us) server_negotiation.cc:407] Connection header received
0620 23:04:51.211067 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:51.211074 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:51.211191 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:51.211315 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:51.212062 (+ 747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48442: BlockingRecv error: recv got EOF from 127.0.0.1:48442 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
I20260620 23:04:51.285537 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:51.279660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47746 (local address 127.2.237.61:40929)
0620 23:04:51.279865 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:51.279872 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:51.280285 (+ 413us) server_negotiation.cc:407] Connection header received
0620 23:04:51.280449 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:51.280455 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:51.280540 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:51.280646 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:51.281036 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:51.282131 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:51.283594 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:51.284247 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:51.284383 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:51.284719 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:51.284850 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:51.284942 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:51.285174 (+ 232us) server_negotiation.cc:299] Negotiation successful
0620 23:04:51.285256 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260620 23:04:51.295228 26038 negotiation.cc:337] Failed RPC negotiation. Trace:
0620 23:04:51.293381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48456 (local address 127.2.237.60:36061)
0620 23:04:51.293614 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:51.293621 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:51.293830 (+ 209us) server_negotiation.cc:407] Connection header received
0620 23:04:51.294012 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:51.294017 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:51.294090 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:51.294180 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0620 23:04:51.294962 (+ 782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48456: BlockingRecv error: recv got EOF from 127.0.0.1:48456 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
I20260620 23:04:51.373827 26016 catalog_manager.cc:1259] Loaded cluster ID: 1ebc5e2167254c3b923754b637f61947
I20260620 23:04:51.373929 26016 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3b3a416e513b449eb77ec0d16de1b301: loading cluster ID for follower catalog manager: success
I20260620 23:04:51.376515 26016 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3b3a416e513b449eb77ec0d16de1b301: acquiring CA information for follower catalog manager: success
I20260620 23:04:51.378328 26016 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3b3a416e513b449eb77ec0d16de1b301: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260620 23:04:51.431674 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:51.426665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47758 (local address 127.2.237.61:40929)
0620 23:04:51.426850 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:51.426854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:51.427129 (+ 275us) server_negotiation.cc:407] Connection header received
0620 23:04:51.427301 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:51.427305 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:51.427376 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:51.427490 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:51.427855 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:51.428595 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:51.429906 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:51.430309 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:51.430409 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:51.430990 (+ 581us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:51.431117 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:51.431216 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:51.431346 (+ 130us) server_negotiation.cc:299] Negotiation successful
0620 23:04:51.431438 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260620 23:04:51.443197 26038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:51.437688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48460 (local address 127.2.237.60:36061)
0620 23:04:51.437903 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:51.437910 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:51.438128 (+ 218us) server_negotiation.cc:407] Connection header received
0620 23:04:51.438289 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:51.438293 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:51.438442 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:51.438539 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:51.438984 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:51.439686 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:51.440926 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:51.441380 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:51.441475 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:51.441961 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:51.442511 (+ 550us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:51.442694 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:51.442797 (+ 103us) server_negotiation.cc:299] Negotiation successful
0620 23:04:51.442862 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260620 23:04:51.467797 26174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:51.448873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43566 (local address 127.2.237.62:43431)
0620 23:04:51.460272 (+ 11399us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:51.460284 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:51.460316 (+ 32us) server_negotiation.cc:407] Connection header received
0620 23:04:51.460481 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:51.460486 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:51.460774 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:51.461078 (+ 304us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:51.461931 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:51.463048 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:51.465189 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:51.465788 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:51.465994 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:51.466764 (+ 770us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:51.467102 (+ 338us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:51.467233 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:51.467345 (+ 112us) server_negotiation.cc:299] Negotiation successful
0620 23:04:51.467432 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11201,"thread_start_us":12375,"threads_started":1}
I20260620 23:04:51.468758 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:51.449196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47768 (local address 127.2.237.61:40929)
0620 23:04:51.449449 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:51.449458 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:51.462232 (+ 12774us) server_negotiation.cc:407] Connection header received
0620 23:04:51.462445 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:51.462452 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:51.462541 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:51.462659 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:51.463417 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:51.464552 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:51.466603 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:51.467265 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:51.467381 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:51.468011 (+ 630us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:51.468131 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:51.468194 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:51.468423 (+ 229us) server_negotiation.cc:299] Negotiation successful
0620 23:04:51.468505 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260620 23:04:51.470351 26038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0620 23:04:51.449236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48470 (local address 127.2.237.60:36061)
0620 23:04:51.449461 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0620 23:04:51.449469 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0620 23:04:51.463089 (+ 13620us) server_negotiation.cc:407] Connection header received
0620 23:04:51.463329 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0620 23:04:51.463336 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0620 23:04:51.463440 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0620 23:04:51.463549 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0620 23:04:51.464227 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:51.465405 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:51.467857 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0620 23:04:51.468502 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0620 23:04:51.468605 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0620 23:04:51.469508 (+ 903us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0620 23:04:51.469636 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0620 23:04:51.469736 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0620 23:04:51.469939 (+ 203us) server_negotiation.cc:299] Negotiation successful
0620 23:04:51.470039 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260620 23:04:51.473078 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 25783
I20260620 23:04:51.474138 25845 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:04:51.588609 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 25783
W20260620 23:04:51.617906 25947 connection.cc:535] server connection from 127.0.0.1:48376 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 23:04:51.618661 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 25857
W20260620 23:04:51.618534 25875 connection.cc:535] server connection from 127.0.0.1:47690 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260620 23:04:51.619863 25919 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:04:51.728417 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 25857
I20260620 23:04:51.748638 2996 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu pid 25931
I20260620 23:04:51.749733 25993 generic_service.cc:196] Checking for leaks (request via RPC)
I20260620 23:04:51.862255 2996 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskWPCN6h/build/asan/bin/kudu with pid 25931
2026-06-20T23:04:51Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4447 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (8885 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (373019 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST